|Dataset Placement and Data Loading Optimizations for Cloud-Native Deep Learning Workloads
The primary challenge facing cloud-based deep learning systems is the need for efficient orchestration of large-scale datasets with diverse data formats and provisioning of high-performance data loading capabilities. To that end, we present DLCache, a cloud-native dataset management and runtime-aware data-loading solution for deep learning training jobs. DLCache supports the low-latency and high-throughput I/O requirements of DL training jobs using cloud buckets as persistent data storage and a dedicated computation cluster for training.
|Year of Publication
IEEE International Symposium on Real-time Computing (ISORC)
|Google Scholar | BibTeX | XML | DOI