Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Deep Learning with C++
  • Table Of Contents Toc
Deep Learning with C++

Deep Learning with C++

By : Bill Chen, Vikash Gupta
5 (1)
close
close
Deep Learning with C++

Deep Learning with C++

5 (1)
By: Bill Chen, Vikash Gupta

Overview of this book

Deep learning systems often struggle to meet performance demands in real-time and production environments. This book shows you how to build high-performance deep learning systems in C++, enabling efficient and scalable artificial intelligence (AI) in resource-constrained environments where performance matters. You’ll start by setting up a complete C++ deep learning environment and implementing core neural networks from scratch. As you progress, you’ll build advanced architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory Networks (LSTMs), Generative Adversarial Networks (GANs), and Transformers, using C++, CUDA, and PyTorch’s C++ API. The book then focuses on model quantization and compression. It will guide you through the model deployment process in production with robust monitoring and explainability. You’ll also explore distributed training and techniques for real-time inference in performance-critical domains. By the end of this book, you’ll be able to design, optimize, and deploy deep learning systems in C++ that are production-ready, scalable, and efficient across multiple industries. *Email sign-up and proof of purchase required
Table of Contents (20 chapters)
close
close
Lock Free Chapter
1
Foundations of Deep Learning in C++
5
Building and Training Neural Networks in C++
12
Deploying, Monitoring, and Explaining Deep Learning Systems in Production
18
Other Books You May Enjoy
19
Index

Deploying and Optimizing Models for Inference

You’ve trained a model that performs brilliantly offline. Now comes the hard part: shipping it—and keeping it fast, reliable, and cost-effective as usage grows. This chapter is about inference in the real world: turning a checkpoint into a predictable C++ service that meets strict latency/throughput Service Level Objectives (SLOs), scales cleanly, and stays healthy over time.

By “inference,” we mean running a trained model to produce predictions; “serving” is exposing that computation behind an API or stream while managing versions, scaling, and observability. We’ll cover the common modes you’ll encounter: online/real-time (low-latency requests), streaming (e.g., token output for LLMs), batch (throughput-first jobs), and edge/on-device (privacy/latency). What matters in production are the SLOs: latency (p50/p95/p99; time-to-first-byte/token), throughput (QPS/tokens-per-second...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Deep Learning with C++
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon