Book Image

Managing Data Science

By : Kirill Dubovikov
Book Image

Managing Data Science

By: Kirill Dubovikov

Overview of this book

Data science and machine learning can transform any organization and unlock new opportunities. However, employing the right management strategies is crucial to guide the solution from prototype to production. Traditional approaches often fail as they don't entirely meet the conditions and requirements necessary for current data science projects. In this book, you'll explore the right approach to data science project management, along with useful tips and best practices to guide you along the way. After understanding the practical applications of data science and artificial intelligence, you'll see how to incorporate them into your solutions. Next, you will go through the data science project life cycle, explore the common pitfalls encountered at each step, and learn how to avoid them. Any data science project requires a skilled team, and this book will offer the right advice for hiring and growing a data science team for your organization. Later, you'll be shown how to efficiently manage and improve your data science projects through the use of DevOps and ModelOps. By the end of this book, you will be well versed with various data science solutions and have gained practical insights into tackling the different challenges that you'll encounter on a daily basis.
Table of Contents (18 chapters)
Free Chapter
1
Section 1: What is Data Science?
5
Section 2: Building and Sustaining a Team
9
Section 3: Managing Various Data Science Projects
14
Section 4: Creating a Development Infrastructure

Storing data along with the code

As you have seen previously, we can structure code in data science projects into a set of pipelines that produce various artifacts: reports, models, and data. Different versions of code produce changing outputs, and data scientists often need to reproduce results or use artifacts from past versions of pipelines.

This distinguishes data science projects from software projects and creates a need for managing data versions along with the code: Data Version Control (DVC). In general, different software versions can be reconstructed by using the source code alone, but for data science projects this is not sufficient. Let's see what problems arise when you try to track datasets using Git.

Tracking and versioning data

...