Presentation at III Multicore World – Sponsored by Catalyst IT
Scalable distributed storage with CEPH
Ricardo Rocha, Cloud Engineer
High Performance Computing (HPC) applications have always imposed significant requirements on data storage systems, both in the amount of data being stored and the need for low latency and high bandwidth for appropriate processing.
In recent years, a significant shift has occurred with storage solutions being offered where new interfaces and paradigms aim at relaxing some of the features of traditional access protocols (strict posix, posix like, …) to achieve greater scalability and availability.
In this presentation we introduce the available interfaces in one of the most popular solutions, the open source CEPH data storage platform. We describe the possibilities for both legacy and new applications to explore these new features in the best way. And we present results from a detailed performance evaluation, covering a comparison for a variety of data access workloads via traditional posix, block level and direct access to its distributed object store.
Ricardo is a cloud engineer at Catalyst IT in Wellington, where he currently focus on automation and scalability of the OpenStack and CEPH infrastructure.
Prior to joining Catalyst he was a software engineer at CERN in Geneva, working on distributed systems and especially on the development of data storage systems in use by the Large Hadron Collider (LHC) grid infrastructure.