Running HEP Workloads on CephFS

Deploy Ceph and baseline the performance on your CloudLab cluster using the directions in the blog: Ansible/Docker: Baselining Ceph on CloudLab. This ensures you have a healthy Ceph setup. After that is done, pull this code and configure the High Energy Physics (HEP) experiment to fit CloudLab, just like you did in the blog (you can even use the same vars.yml and hosts configurations from your baseline experiment):

$ git clone --recursive https://github.com/michaelsevilla/paulocto-popper.git
$ cd paulocto-popper/pipelines/hep/
$ vim hosts
$ vim vars.yml

For this experiment, we also need to provide a ROOT file and a trace of NTupleMaker. These are large files, so create space for them on the CloudLab head node using the extra disk:

$ sudo mkfs.ext4 /dev/sdb; sudo mkdir /mnt/disk;
$ sudo mount /dev/sdb /mnt/disk/
$ sudo mkdir /mnt/disk/root; sudo chmod 777 -R /mnt/disk/root

The directory on the CloudLab head node (/mnt/disk/root) should match the directory in vars.yml of the HEP experiment directory. Then, from your laptop:

$ scp 2AC36403-8E7E-E711-A599-02163E01366D.root <IP0>:/mnt/disk/root/
$ scp cmsdump.outerr <IP0>:/mnt/disk/root/

Back on the head node, run the setup script on both the server and the client. This isn’t included as Ansible code because the root code is in a private repository. The <CLIENT IP> should match the client host in your hosts directory:

$ ./setup_client.sh <GITHUB USERNAME> <GITHUB PASSWORD> <CLIENT IP>
$ ./setup.sh <GITHUB USERNAME> <GITHUB PASSWORD>

The ./setup.sh script takes a long time because we generate the trace (i.e. list of branch accesses) from the ROOT file and the cmsdump.outerr file. Now we are ready to run the experiment:

$ ./run.sh

Results are in the results-run directories.

Jekyll theme inspired by researcher