Book Image

Bioinformatics with Python Cookbook - Second Edition

By : Tiago Antao
Book Image

Bioinformatics with Python Cookbook - Second Edition

By: Tiago Antao

Overview of this book

Bioinformatics is an active research field that uses a range of simple-to-advanced computations to extract valuable information from biological data. This book covers next-generation sequencing, genomics, metagenomics, population genetics, phylogenetics, and proteomics. You'll learn modern programming techniques to analyze large amounts of biological data. With the help of real-world examples, you'll convert, analyze, and visualize datasets using various Python tools and libraries. This book will help you get a better understanding of working with a Galaxy server, which is the most widely used bioinformatics web-based pipeline system. This updated edition also includes advanced next-generation sequencing filtering techniques. You'll also explore topics such as SNP discovery using statistical approaches under high-performance computing frameworks such as Dask and Spark. By the end of this book, you'll be able to use and implement modern programming techniques and frameworks to deal with the ever-increasing deluge of bioinformatics data.

Related Content you might be interested in

Current Title:

Small book image
Bioinformatics with Python Cookbook - Second Edition
Tiago Antao
Nov, 2018 | 360 pages
No titles found
Table of Contents (16 chapters)
Title Page
Title Page
About Packt
About Packt
Contributors
Contributors
Preface
Preface
Free Chapter
1
Python and the Surrounding Software Ecology
Python and the Surrounding Software Ecology
Introduction
Installing the required software with Anaconda
Installing the required software with Docker
Interfacing with R via rpy2
Performing R magic with Jupyter Notebook
2
Next-Generation Sequencing
Next-Generation Sequencing
Introduction
Accessing GenBank and moving around NCBI databases
Performing basic sequence analysis
Working with modern sequence formats
Working with alignment data
Analyzing data in VCF
Studying genome accessibility and filtering SNP data
Processing NGS data with HTSeq
3
Working with Genomes
Working with Genomes
Introduction
Working with high-quality reference genomes
Dealing with low-quality genome references
Traversing genome annotations
Extracting genes from a reference using annotations
Finding orthologues with the Ensembl REST API
Retrieving gene ontology information from Ensembl
4
Population Genetics
Population Genetics
Introduction
Managing datasets with PLINK
Introducing the Genepop format
Exploring a dataset with Bio.PopGen
Computing F-statistics
Performing Principal Components Analysis
Investigating population structure with admixture
5
Population Genetics Simulation
Population Genetics Simulation
Introduction
Introducing forward-time simulations
Simulating selection
Simulating population structure using island and stepping-stone models
Modeling complex demographic scenarios
6
Phylogenetics
Phylogenetics
Introduction
Preparing a dataset for phylogenetic analysis
Aligning genetic and genomic data
Comparing sequences
Reconstructing phylogenetic trees
Playing recursively with trees
Visualizing phylogenetic data
7
Using the Protein Data Bank
Using the Protein Data Bank
Introduction
Finding a protein in multiple databases
Introducing Bio.PDB
Extracting more information from a PDB file
Computing molecular distances on a PDB file
Performing geometric operations
Animating with PyMOL
Parsing mmCIF files using Biopython
8
Bioinformatics Pipelines
Bioinformatics Pipelines
Introduction
Introducing Galaxy servers
Accessing Galaxy using the API
Developing a Galaxy tool
Using generic pipelines with bioinformatics data
Deploying a variant analysis pipeline with Airflow
9
Python for Big Genomics Datasets
Python for Big Genomics Datasets
Introduction
Using high-performance data formats – HDF5
Doing parallel computing with Dask
Using high-performance data formats – Parquet
Computing sequencing statistics using Spark
Optimizing code with Cython and Numba
10
Other Topics in Bioinformatics
Other Topics in Bioinformatics
Introduction
Doing metagenomics with the QIIME 2 Python API
Inferring shared chromosomal segments with Germline
Accessing the Global Biodiversity Information Facility via REST
Georeferencing GBIF datasets
Plotting protein interactions with Cytoscape the hard way
11
Advanced NGS Processing
Advanced NGS Processing
Introduction
Preparing the dataset for analysis
Using Mendelian error information for quality control
Using decision trees to explore the data
Exploring the data with standard statistics
Finding genomic features from sequencing annotations
Index
Index

Computing sequencing statistics using Spark

If you need to use parallel computing, then Spark is one alternative to Dask. Its abstraction level is slightly higher. This gives you less granular control over the computation, but is more declarative to code. Spark is also somewhat language agnostic (it is actually Java/Scala-based). Here, we will compute some very basic statistics over the Parquet dataset that we generated in the previous recipe.

Getting ready

Preparing for this recipe can be quite tricky. First, we will have to start a Spark server. At the time of writing this book, the conda packages for accessing Spark were quite immature. We will still use conda here, but we will not install any Spark packages from conda. Follow these steps to prepare the environment:

  1. Make sure that you have Java 8 installed. Be careful with the Java version, as an older version will not work, but a newer might also be problematic.
  2. Download Spark (https://spark.apache.org/downloads.html). This code was tested...
Previous Section
End of Section 6
Next Section