Preface

Optimus is a Python library that works as a unified API for data cleaning, processing, and merging. It can be used for small and big data on local and big clusters using CPUs or GPUs. Data Processing with Optimus shows you how to use the library to enhance your data science workflow.

The book begins by covering the internals of Optimus and showing you how it works in tandem with existing technologies to serve users' data processing needs. You'll then use Optimus to load and save data from text data formats such as CSV and JSON files, explore binary files such as Excel, and for columnar data processing with Parquet, Avro, and OCR. Next, you'll learn about the profiler and profiler data types, a unique feature of Optimus DataFrames that helps you get an overview of the data quality in every column. You'll also create data cleaning and transformation functions and add a hypothetical new data processing engine. Later, you'll explore plots in Optimus such as histograms and box plots, and learn how Optimus lets you connect to any other library, including Plotly and Altair. Finally, you'll understand the advanced applications of Optimus, such as feature engineering, machine learning, and NLP, along with exploring the advancements in Optimus.

By the end of this book, you'll be able to easily improve your data science workflow with Optimus.