Today’s storage systems are designed to be treated as monolothic, so they do not expose their internal functionality and instead give applications a simple file, block, or object interface. Applications that need similar functionality cannot access the storage system’s internal implementation, so they end up doing their own data management by bolting on more code or re-implementing functionality that can be found lower in the stack.

A programmable storage system exposes internal subsystem abstractions as “interfaces” to enable the creation of higher-level services via composition. Malacology is a programmable storage system that enables the programmability of internal abstractions in Ceph. Using Malacology, we built the Mantle and ZLog services.

Malacology

News

Dec 2019. SkyhookDM to appear at Vault’20! “Scaling Databases and File APIs with Programmable Ceph Object Storage”.
Nov 2019. SkyhookDM presented at the International Parallel Data Systems Workshop (PDSW19) co-located with Supercomputing 2019 in Denver, CO. “Towards Physical Design Management in Storage Systems”. Full publication.
Nov 2019. SkyhookDM presented at 24th International Conference on Computing in High Energy and Nuclear Physics in Adelaide, Australia. “Mapping Scientific Datasets to Programmable Object Storage”.
Oct 2019. SkyhookDM presented at the National Diversity in STEM Conference (SACNAS) in Honolulu, Hawai’i. “Helping Scientists Fly Over Data without Getting Swamped”.
Oct 2019. Ashay presents his GSoC project and Jeff presents SkyhookDM during the storage systems track at the 4th Annual CROSS Symposium in Santa Cruz, CA.
Aug 2019. GSoC project successfully completed, please checkout the report here. Ashay embedded column-oriented data formats and processing with Apache Arrow within our SkyhookDM Ceph extensions. Thank you for a great project Ashay.
May 2019. SkyhookDM welcomes our first GSoC student, Ashay Shirwadkar!
March 2019. SkyhookDM has potential projects posted for Google Summer of Code 2019! CROSS was accepted again this year as a GSoC mentor organization and has other projects posted as well, please take a look. And see our Gitter channel to interact directly with mentors. Applications are open until April 9, 2019. More info here.
Feb 2019. SkyhookDM presented at Vault’19! The Linux Storage and File Systems Conference (co-located with FAST’19) in Boston, MA. (pdf slides)
11/26/18: New web page for Programmable Storage for Databases (SkyhookDM) [SkyhookDM.com]
09/05/18: New NSF Award to help establish IRIS-HEP! [link]
08/23/18: New NSF Award for Declarative Programmable Storage! [link]
07/09/18: Tintenfisch (follow up to Cudele) published at HotStorage’18! [link]
06/19/18: Congratulations, Dr. Watkins! [link]
04/25/18: Congratulations, Dr. Sevilla! [link]
04/24/18: “Zlog Entry Caching & Benchmarking” selected as GSoC project! [link]
01/29/18: Congratulations, Dr. Ionkov! [link]
01/29/18: Programmable Caches (follow up to Mantle) published at CCGrid 2018! [link]
01/29/18: Cudele published at IDPPS 2018! [link]
12/18/17: Publication on collaboration with Stanford and LANL published at HiPC 2017! [link]
11/13/17: Asgard (related to Malacology) published at PDSW-DISCS 2017! [link]
08/01/17: Malacology featured in The Next Platform magazine! [link]
07/10/17: DeclStor (follow up to Malacology) published at HotStorage 2017! [link]
07/07/17: New NSF Award for Programmable Storage! [link]
04/23/17: Malacology published at EuroSys 2017! [link]
03/22/17: Mantle and ZLog funding renewed by CROSS! [link]
10/26/16: Mantle merged into Ceph! [link]
10/05/16: ZLog poster published at SoCC 2016! [link]
09/22/16: Lua CLS (Malacology interface) merged into Ceph! [link]
01/15/16: Mantle funded by CROSS! [link]
11/15/15: Mantle published at SC 2015! [link]
09/25/15: ZLog funded by CROSS! [link]

Researchers

Faculty: Carlos Maltzahn, Peter Alvaro, Shel Finkelstein, Jeff LeFevre, Ike Nassi

{carlosm, palvaro, shel, jlefevre,inassi}@ucsc.edu
Research Scientists: Ivo Jimenez

ivotron@ucsc.edu
Students: Xiaowei (Aaron) Chu, Jianshen Liu, Esmaeil Mirvakili, Yiming Zhang

{xweichu,jliu120,smirvaki,yzhan298}@ucsc.edu
Alumni: Ivo Jimenez, Noah Watkins, Michael Sevilla, Latchesar Ionkov

ivotron@ucsc.edu, noahwatkins@gmail.com, msevilla@ucsc.edu, lionkov@lanl.gov

Jekyll theme inspired by researcher