Architecture · Storage · Scale
Designing a Memory-Efficient Data Distribution Architecture
I started by investigating OOM failures in a data-export path. Tracing the data further exposed the same full-dataset assumption on the serving side, which led me to evaluate alternative designs and prototype a local LMDB snapshot approach.