-
Book Overview & Buying
-
Table Of Contents
Deep Learning with C++
By :
When datasets outgrow RAM, three patterns help: memory mapping, batch (streaming) I/O, and distributed filesystems. Here we focus on the first two because they're portable and easy to demonstrate in standalone C++; distributed filesystems require cluster infrastructure and are beyond this book's scope. For a runnable example that the snippets come from, see bigdata_demo.cpp.
Memory-mapped I/O lets you treat a file as a contiguous block of memory, avoiding explicit read loops and extra copies—ideal for fast random access over large datasets. The RAII wrapper below opens a file read-only, maps it into the process address space, and ensures it is cleanly unmapped/closed on destruction (POSIX mmap; on Windows use CreateFileMapping/MapViewOfFile).
struct MappedFile {
void* data=nullptr; size_t size=0; int fd=-1;
explicit MappedFile(const std::string& path) {
fd = ::open(path.c_str(), O_RDONLY);
if (fd < 0) throw...