Book Image

Bioinformatics with Python Cookbook - Second Edition

By : Tiago Antao
Book Image

Bioinformatics with Python Cookbook - Second Edition

By: Tiago Antao

Overview of this book

Bioinformatics is an active research field that uses a range of simple-to-advanced computations to extract valuable information from biological data. This book covers next-generation sequencing, genomics, metagenomics, population genetics, phylogenetics, and proteomics. You'll learn modern programming techniques to analyze large amounts of biological data. With the help of real-world examples, you'll convert, analyze, and visualize datasets using various Python tools and libraries. This book will help you get a better understanding of working with a Galaxy server, which is the most widely used bioinformatics web-based pipeline system. This updated edition also includes advanced next-generation sequencing filtering techniques. You'll also explore topics such as SNP discovery using statistical approaches under high-performance computing frameworks such as Dask and Spark. By the end of this book, you'll be able to use and implement modern programming techniques and frameworks to deal with the ever-increasing deluge of bioinformatics data.

Related Content you might be interested in

Current Title:

Small book image
Bioinformatics with Python Cookbook - Second Edition
Tiago Antao
Nov, 2018 | 360 pages
No titles found
Table of Contents (16 chapters)
Title Page
Title Page
About Packt
About Packt
Contributors
Contributors
Preface
Preface
Free Chapter
1
Python and the Surrounding Software Ecology
Python and the Surrounding Software Ecology
Introduction
Installing the required software with Anaconda
Installing the required software with Docker
Interfacing with R via rpy2
Performing R magic with Jupyter Notebook
2
Next-Generation Sequencing
Next-Generation Sequencing
Introduction
Accessing GenBank and moving around NCBI databases
Performing basic sequence analysis
Working with modern sequence formats
Working with alignment data
Analyzing data in VCF
Studying genome accessibility and filtering SNP data
Processing NGS data with HTSeq
3
Working with Genomes
Working with Genomes
Introduction
Working with high-quality reference genomes
Dealing with low-quality genome references
Traversing genome annotations
Extracting genes from a reference using annotations
Finding orthologues with the Ensembl REST API
Retrieving gene ontology information from Ensembl
4
Population Genetics
Population Genetics
Introduction
Managing datasets with PLINK
Introducing the Genepop format
Exploring a dataset with Bio.PopGen
Computing F-statistics
Performing Principal Components Analysis
Investigating population structure with admixture
5
Population Genetics Simulation
Population Genetics Simulation
Introduction
Introducing forward-time simulations
Simulating selection
Simulating population structure using island and stepping-stone models
Modeling complex demographic scenarios
6
Phylogenetics
Phylogenetics
Introduction
Preparing a dataset for phylogenetic analysis
Aligning genetic and genomic data
Comparing sequences
Reconstructing phylogenetic trees
Playing recursively with trees
Visualizing phylogenetic data
7
Using the Protein Data Bank
Using the Protein Data Bank
Introduction
Finding a protein in multiple databases
Introducing Bio.PDB
Extracting more information from a PDB file
Computing molecular distances on a PDB file
Performing geometric operations
Animating with PyMOL
Parsing mmCIF files using Biopython
8
Bioinformatics Pipelines
Bioinformatics Pipelines
Introduction
Introducing Galaxy servers
Accessing Galaxy using the API
Developing a Galaxy tool
Using generic pipelines with bioinformatics data
Deploying a variant analysis pipeline with Airflow
9
Python for Big Genomics Datasets
Python for Big Genomics Datasets
Introduction
Using high-performance data formats – HDF5
Doing parallel computing with Dask
Using high-performance data formats – Parquet
Computing sequencing statistics using Spark
Optimizing code with Cython and Numba
10
Other Topics in Bioinformatics
Other Topics in Bioinformatics
Introduction
Doing metagenomics with the QIIME 2 Python API
Inferring shared chromosomal segments with Germline
Accessing the Global Biodiversity Information Facility via REST
Georeferencing GBIF datasets
Plotting protein interactions with Cytoscape the hard way
11
Advanced NGS Processing
Advanced NGS Processing
Introduction
Preparing the dataset for analysis
Using Mendelian error information for quality control
Using decision trees to explore the data
Exploring the data with standard statistics
Finding genomic features from sequencing annotations
Index
Index

Deploying a variant analysis pipeline with Airflow

Here, we will develop a mini variant analysis pipeline with Airflow. The objective here is not to get the scientific part right—we cover that in other chapters—but to see how to create components with Airflow. Our mini-pipeline will download HapMap data, sub-sample at 1% and 10%, do a simple PCA, and draw it.

 

Getting ready

You will need PLINK installed. Remember that we are not using a conda environment, so you have to make sure it is available for Airflow. We will define the following tasks:

  1. Downloading data
  2. Uncompressing it
  3. Sub-sampling at 10%
  4. Sub-sampling at 1%
  5. Computing PCA on the 1% sub-sample
  6. Charting the PCA

Our pipeline recipe will have two parts: the actual coding of the pipeline and making the pipeline actually execute.

The code for this can be found on Chapter08/pipelines/airflow/create_tasks.py.

How to do it…

  1. With Airflow, a pipeline specification is written in Python, which is quite convenient for us! Let's start with the import part:
from...
Previous Section
End of Chapter 8
Next Chapter