storage systems

Project Idea Description Topics: Storage systems, machine learning Skills: Python, PyTorch, Bash scripting, Linux, Machine Learning modeling Difficulty: Hard Size: Large (350 hours) Mentors: Yuyang (Roy) Huang (primary contact), Swami Sundararaman Contributor(s): Lihaowen (Jayce) Zhu In the realm of machine learning (ML), preprocessing of data is a critical yet often underappreciated phase, consuming approximately 80% of the time in common ML tasks.

Project Idea Description Topics: Storage Systems, Machine Learning, Erasure Coding Skills: C/C++, Python, PyTorch, Bash scripting, Linux, Erasure Coding, Machine Learning Difficulty: Hard Size: Large (350 hours) Mentors: Meng Wang (primary contact), John Bent Large-scale data centers store immense amounts of user data across a multitude of disks, necessitating redundancy strategies like erasure coding (EC) to safeguard against disk failures.

Project Idea Description Topics: Storage systems, machine learning Skills: Python, PyTorch, Bash scripting, Linux, Machine Learning modeling Difficulty: Hard Size: Large (350 hours) Mentors: Ruidan Li (primary contact), Kexin Pei In the realm of modern applications, achieving not only low but also predictable response times is a critical requirement.

Project Idea Description Topics: Storage Systems, Erasure Coding Skills: C/C++, Java, Bash scripting, Linux, HDFS, ZFS, Erasure Coding Difficulty: Hard Size: Large (350 hours) Mentors: Meng Wang (Main contact person) and Anjus George Multi-Level Erasure Coding (MLEC), which performs erasure coding at both network and local levels, has seen large deployments in practice.

Project Idea Description Topics: Storage systems, machine learning Skills: C/C++, Python, PyTorch, Bash scripting, Linux, Machine Learning modeling Difficulty: Hard Size: Large (350 hours) Mentors: Daniar H. Kurniawan (primary contact), Haryadi Gunawi The contemporary landscape of high-performance servers, particularly those designed for data centers and AI/ML training, prominently features solid-state drives (SSDs) and spinning disks (HDDs) as primary storage devices.

The Data Storage Research Vision 2025, organized in an NSF workshop, calls for more “AI for storage” research. However, performing ML-for-storage research can be a daunting task for new storage researchers.

Massive storage systems rely heavily on erasure coding (EC) to protect data from drive failures and provide data durability. Existing storage systems mostly adopt single-level erasure coding (SLEC) to protect data, either performing EC at the network level or performing EC at the local level.

CephFS is a distributed file system on top of Ceph. It is implemented as a distributed metadata service (MDS) that uses dynamic subtree balancing to trade parallelism for locality during a continually changing workloads.

HDF5 is a unique technology suite that makes possible the management of extremely large and complex data collections. The HDF5 technology suite includes: A versatile data model that can represent very complex data objects and a wide variety of metadata.