Book Image

Bioinformatics with Python Cookbook - Second Edition

By : Tiago Antao
Book Image

Bioinformatics with Python Cookbook - Second Edition

By: Tiago Antao

Overview of this book

Bioinformatics is an active research field that uses a range of simple-to-advanced computations to extract valuable information from biological data. This book covers next-generation sequencing, genomics, metagenomics, population genetics, phylogenetics, and proteomics. You'll learn modern programming techniques to analyze large amounts of biological data. With the help of real-world examples, you'll convert, analyze, and visualize datasets using various Python tools and libraries. This book will help you get a better understanding of working with a Galaxy server, which is the most widely used bioinformatics web-based pipeline system. This updated edition also includes advanced next-generation sequencing filtering techniques. You'll also explore topics such as SNP discovery using statistical approaches under high-performance computing frameworks such as Dask and Spark. By the end of this book, you'll be able to use and implement modern programming techniques and frameworks to deal with the ever-increasing deluge of bioinformatics data.
Table of Contents (16 chapters)
Title Page
About Packt
Contributors
Preface
Index

Doing parallel computing with Dask


The previous code is still quite slow, so now, we will use parallel processing to accelerate our data analysis. Our first approach will be using Dask, a Python-based library that provides scalable parallelism: most of the code that scales on your laptop will be able to scale on a large cluster. Dask is a fairly low-level and Python-related approach. Later in this chapter, we will discuss an alternative approach that is more high-level and language-agnostic.

Getting ready

We will make a parallel version of the previous code, so you will need to have the same dataset available. We will be using HDF5 processing, so you should be acquainted with the previous recipe anyway.

How to do it...

Take a look at the following steps:

  1. We will start by doing the necessary imports and checking Dask's version:
from multiprocessing.pool import Pool
from math import ceil

import numpy as np

import h5py

import dask
import dask.array as da
import dask.multiprocessing
print(dask...