Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Data Engineering Best Practices
  • Table Of Contents Toc
Data Engineering Best Practices

Data Engineering Best Practices

By : Richard J. Schiller, David Larochelle
5 (2)
close
close
Data Engineering Best Practices

Data Engineering Best Practices

5 (2)
By: Richard J. Schiller, David Larochelle

Overview of this book

Revolutionize your approach to data processing in the fast-paced business landscape with this essential guide to data engineering. Discover the power of scalable, efficient, and secure data solutions through expert guidance on data engineering principles and techniques. Written by two industry experts with over 60 years of combined experience, it offers deep insights into best practices, architecture, agile processes, and cloud-based pipelines. You’ll start by defining the challenges data engineers face and understand how this agile and future-proof comprehensive data solution architecture addresses them. As you explore the extensive toolkit, mastering the capabilities of various instruments, you’ll gain the knowledge needed for independent research. Covering everything you need, right from data engineering fundamentals, the guide uses real-world examples to illustrate potential solutions. It elevates your skills to architect scalable data systems, implement agile development processes, and design cloud-based data pipelines. The book further equips you with the knowledge to harness serverless computing and microservices to build resilient data applications. By the end, you'll be armed with the expertise to design and deliver high-performance data engineering solutions that are not only robust, efficient, and secure but also future-ready.
Table of Contents (21 chapters)
close
close

Test framework example

Now that you have become aware of some of the approaches to testing, it’s time to look at a practical example and for that, we have to begin with key drivers. Please look at Figure 10.1, where you can see a test framework that you can take into account as you develop your solution. It should be formalized and made available to all those on the solution’s various project teams to utilize. In Figure 10.1 you can see that your test framework, integration, and best practices work together to form the scope of your test effort:

Figure 10.1 – Azure DevOps scope

Figure 10.1 – Azure DevOps scope

Remember that the goal is to reduce the overhead of testing and make it as easy as possible to run tests. The developers and testers need to be able to catch deviations from the status quo as development sprints release code into various developed environments. You will know that the system is minimally up and running based on integration tests that are running...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Data Engineering Best Practices
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon