OpenStack Liberty Design Summit has ended
Tuesday, May 19 • 11:15am - 11:55am
Ceph at CERN: A Year in the Life of a Petabyte-Scale Block Storage Service

Sign up or log in to save this to your schedule and see who's attending!

The CERN IT Department operates a large distributed OpenStack infrastructure with around 10,000 VMs and 100,000 CPU cores. This cloud is predominantly used for physics data analysis, but there remains a long tail of conventional IT services and user-managed application VMs. These varied use-cases created a new requirement in our department, one to build a scalable, reliable, performant and future-proof block storage backend. To solve this problem, in mid-2013 we deployed 3 petabytes of Ceph software-defined storage.

This talk will present a year in the life of our Ceph cluster, focusing on the operations issues related to the RBD use-cases. Based on our experience hosting around 1000 Cinder volumes and 1500 Glance images, we’ll describe how the cluster is used, its access patterns and growth, mistakes we've made, problems overcome, upgrades successfully performed, and highlight some tips and tricks we’ve learned to tune both the servers (Ceph) and clients (qemu-kvm). Lastly, we’ll conclude by outlining our future plans for cloud-based block storage across our multiple datacentres and describe where we believe cloud storage may have further potential at CERN.


Tuesday May 19, 2015 11:15am - 11:55am
Room 109

Attendees (0)