Docker: Distributing Ceph Images

You have made a Docker image from the Docker: Building Ceph Images blog and now you want to share it with your cluster.

Using Dockerhub

Dockerhub is public and adheres to the Popper Convention.

1. Tag your image:

   bash docker tag mantle:latest michaelsevilla/mantle:jewel

2. Login to Dockerhub:

   bash $ docker login ... Login Succeeded

3. Push your image:

   docker push michaelsevilla/mantle:jewel
   

Using an Internal Registry

An in-house Docker image registry is faster than Dockerhub but it’s much harder to Popperize.

1. Start up the registry:

   bash docker run -d \ -p 5000:5000 \ --restart=always \ --name registry \ -v $PWD/data:/var/lib/registry \ registry:2

2. Tag the image:

   bash docker tag mantle:latest $REGISTRY_IP:5000/michaelsevilla/mantle:jewel

3. Push your image to the registry:

   bash docker push $REGISTRY_IP:5000/michaelsevilla/mantle:jewel

   If you get an error:

   bash $ docker push $REGISTRY_IP:5000/michaelsevilla/mds-reqlatency The push refers to a repository [192.168.140.2:5000/michaelsevilla/mds-reqlatency] Get https://192.168.140.2:5000/v1/_ping: tls: oversized record received with length 20527

   You might have to instruct the daemon to use an insecure registry in /etc/docker/daemon.json:

   vim { "insecure-registries": ["192.168.140.2:5000"] }

4. Check out your image sitting in the registry:

   bash $ curl -X GET http://$REGISTRY_IP:5000/v2/_catalog | python -m json.tool { "repositories": [ "michaelsevilla/mantle" ] }

5. On all nodes, add this to the end of /etc/default/docker:

   vim DOCKER_OPTS=--insecure-registry=$REGISTRY_IP:5000

Now we want to distribute the image on the cluster. We could push it to DockerHub but this is slow. Instead we set up our own in-house registry that houses and serves Docker images. For directions for pushing images to DockerHub and setting up your own registry, see our Docker Ceph builder wiki. Assuming you have set up your own registry, we can now push to it:

~/ceph$ docker tag myimg piha.soe.ucsc.edu:5000/ceph/daemon:master
~/ceph$ docker push piha.soe.ucsc.edu:5000/ceph/daemon:master
~/ceph$ curl -X GET http://$REGISTRY_IP:5000/v2/_catalog | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   383  100   383    0     0  15147      0 --:--:-- --:--:-- --:--:-- 15320
{
    "repositories": [
        "ceph/daemon",
    ]
}

Now that we have our image in the registry, if we do a Docker pull on each node we will have an updated image. This works especially well if we are actively developing because new nodes will only pull binaries that differ from the binaries in their images; so if you only touched something in the metadata server only the binaries for the MDS daemon will need to be pulled. More information on how our builder does this with Docker layers can be found in our Docker Ceph builder wiki.

Jekyll theme inspired by researcher