Docker Exercises

License: Creative Commons BY-NC 3.0

Part 2. Containers

Check that docker is correctly running and that you have permission to use the engine.
```
docker info
```
(pull) Pull an image from the official registry, eg: debian:latest (you can browse https://hub.docker.com/ if you want to find other images).
```
docker pull debian:latest
```
debian is the repository name, and :latest is a tag that identifies an image in the repository (for the case of the debian repository it is the latest stable release). You could also write :trixie or :bookworm to use a specific version of this distribution.

You can check that your image is present in the docker engine:
```
docker images
```
(run) Run a container from this image.
```
docker run debian:latest
```
(you may also write docker run debian which is equivalent: :latest is the default tag if none is provided).

Nothing happens? actually the container has already terminated, you can display it with docker ps, but add -a/--all because non-running (stopped) containers are not displayed by default.
```
docker ps -a
```
The default command of the debian image is /bin/bash and by default docker containers are run without stdin (it is redirected from /dev/null). Thus bash exits immediately.
(run a command) You may override the default command by providing extra arguments after the image name. Then this command will be executed (instead of bash).
```
docker run debian ls /bin
docker run debian cat /etc/motd
```
(stdin) Let's go back to bash, this time we want to interact with the shell. To keep stdin open, we launch the container with -i/--interactive.
```
docker run -i debian
```
The container runs, but displays nothing. Actually bash is running in batch mode. You can try to execute commands (eg: ls, id, hostname...) and you will see the result.

Bash is in batch mode because it is not running on a terminal (its stdout is a pipe, not a tty).
(tty) To have a real interactive shell inside our container, we need to allocate a tty with -t/--tty.
```
docker run -t -i debian
```
(start) You can exit your container and display the list of all containers:
```
docker ps -a
```
It is possible to start them again with docker start. Like the run command, you may use -i to have stdin open. Note that start expects you to tell which container you want to start. Containers may be identified either by their id (first column of docker ps) or by their name (last column). You may provide only the first digits of the id (as long as there is no ambiguity). Examples:
```
docker start -i 85bcdca6c38f
docker start -i 85bcd
docker start -i 85
docker start -i 85bcdca6c38f07e3f8140cbf8b4ad37fd80d731b87c6945012479439a450a443
docker start -i pensive_hodgkin
```
(commit) You can modify files inside a container. If you restart the same container you can note that these changes are still present. However they will not be present in the other container (even if they are running the same image) because docker uses a copy-on-write filesystem. Use the command docker diff to show the difference between a container and its original image.

Remember that all changes inside a container are thrown away when the container is removed. If we want to save a container filesystem for later use, we have to commit the container (i.e take a snapshot).
```
docker commit CONTAINER
```
This operation creates a new image (visible in docker images). This image in turn can be used to start a new container.

Note: docker commit does not affect the state of the container. If it is running, then it just keeps running. You may take as many snapshots as you like.
(rm) You now have too many dead containers in your engine. You should use docker rm to remove them. Alternatively you can run docker container prune which removes all dead containers.
(extras) If you still have extra time, you can experiment
- the other docker run options we introduced so far:
  - --rm to remove the container automatically when it terminates
  - -d/--detach to run a container in the background
  - -u/--user to run the container as a different user
  - -w/--workdir to start the container in a different directory
  - -e/--env to set an environment variable
  - -h/--hostname to set a different hostname (the host name inside the container)
  - --name to set a different name (the name of the container in the docker engine)
  - also you may type docker run --help to display all configuration keys
- other docker commands (note: some of these commands require the container to be running, just launch docker run -d -t -i debian to have one that keeps running in the background)
  - docker inspect to display the metadata of a container (json format)
  - docker cp to transfer files from/into the container
  - docker exec to launch a separate command (very useful for providing a debugging shell -> docker exec -t -i CONTAINER bash)
  - docker top to display the processes running inside the container
  - docker stats to display usage statistics
  - docker logs to display the container output
  - docker attach to reattach to the console of a detached container

Part 3. Container inputs/outputs

(external volume) Run a container with -v/--volume to mount an external volume.

Eg: mount the /tmp/myvol from the host machine at /myvol inside the container:
```
docker run --rm -t -i -v /tmp/myvol:/myvol debian
```
Note: on Windows/MacOS over the Docker Desktop, your docker engine is running inside a virtual machine. This means /tmp/myvol refers to the /tmp/myvol path inside the VM. You can mount directories from your host system only if they are shared with the VM. By default, the toolbox is configured to share the Users directory inside the VM:
- on Windows C:\\Users is mounted as /c/Users
- on MacOS /Users is mounted as /Users
Thus you can mount a directory from these places, eg:

docker run --rm -t -i -v '/c/Users/NAME/My documents/myvol:/myvol' debian

docker run --rm -t -i -v '/Users/NAME/Documents/myvol:/myvol' debian
Once the container is started, you can note that the directory /tmp/myvol is mounted inside the container at /myvol. If you create files there they will be visible on both sides, and they will persist if the container is removed. You can remove your container and create a new one with the same parameters to check that.

This way of using an external volume is a direct mount. The docker engine will not care about the management of this directory (apart from creating it).
(named volume) Alternatively we can create a named volume. That is a volume managed by docker (and stored by default in /var/lib/docker/volumes). A named volume is a volume that does not start with /. Example:
```
docker run --rm -t -i -v my-named-volume:/myvol debian
```
This named volume is persistent of course. It is managed separately from the containers with the docker volume command, eg:
```
docker volume ls
docker volume rm my-named-volume
```
(running a server) We will now play with the network.

Pull the nginx:stable-alpine image (nginx is a web server, and alpine is a very lightweight linux distribution based on the musl C library+busybox and a popular solution for building small docker images).
```
docker pull nginx:stable-alpine
```
Our goal here is to run a http server to serve some content (ex: the package documentations on our machine in /usr/share/doc). Before starting the container we need to know how to configure it, especially we need to know where the served directory must be mounted.

The documentation of the nginx image says the content is to be located in /usr/share/nginx/html. So let's mount our doc directory at this place. Note that nginx does not require write access on these files, therefore it is a good idea to append :ro to make the mount read-only.
```
docker run -d --name nginx -v /usr/share/doc:/usr/share/nginx/html:ro nginx:stable-alpine
```
The server is now running somewhere in a container. Since this container has a separate network stack, it has a different IP address. There are multiple ways to obtain this IP address:
- inspect the container metadata:
```
docker inspect nginx | grep IPAddress
```
- run a command inside the container:
```
docker exec nginx ip addr show dev eth0
```
Once you know this address you can open it in your web browser -> http://172.17.x.x/.

Unfortunately the nginx image is compiled without the autoindex module so it will not display directories without an index.html file, and it will return a 403 Forbidden error.

Just find an html document with the following command and append it to your url to see if it works.
```
(cd /usr/share/doc && find * -name index.html)
```
Example: http://172.17.X.X/python3/index.html.
(publish) We have confirmed that we are able to run an HTTP server inside a container and serve some content. However this container is in a private network (e.g., 172.17.0.2/16), it is not reachable from the public.

To make it reachable, we have to publish the HTTP port (tcp/80) on host machine (which may have a public IP address). We add a -p/--publish option:
```
docker run -d --name nginx -v /usr/share/doc:/usr/share/nginx/html:ro -p 80:80 nginx:stable-alpine
```
Then the server should be reachable at http://localhost/. Note: the command will fail if you already have a server using the port 80 of your machine. If this happens, you may specify an alternate port, eg: -p 1234:80 the server will be reachable at http://localhost:1234/

For instance, you should be able to access http://localhost/python3/index.html.
(user-defined network) Our nginx server is reachable from outside. Another use case would be to make a containerized server reachable from another container (for example, a web application in a server may want to use a database hosted in another container).

This is possible by having the two containers connected to the same internal network.

By default, containers are launched on the main bridge network. However, in a production context, to improve the security, it is preferable to put unrelated containers in separate networks.

Note: Depending on the configuration of your daemon (in /etc/docker/ daemon.json), inter-container communications may or may not be authorized on the default bridge (in case we have icc: false). If icc is enabled, containers on the default bridge can already communicate, but only using their IP address: the default bridge does not provide name resolution. Using the container name as TCP/IP destination is only possible on a user-defined network, like the one we create below.

To test this feature we run a busybox container (it's lightweight and it provides the wget http client) and we will create a dedicated network named ngnet, to let our two containers communicate privately.
```
docker network create ngnet
```
We can display its config (and especially observe that it uses a different IP prefix) with:
```
docker network inspect ngnet
```
and
```
docker network inspect ngnet | grep Subnet
```
At container creation time, we can use --network to select a specific network (instead of the default bridge). But it is also possible to connect and disconnect dynamically the containers to networks (especially to allow having a container connected to multiple networks).

We start by connecting our running nginx container to the ngnet network:
```
docker network connect ngnet nginx
```
The container is now connected to the two networks (bridge on eth0 and ngnet on eth1). We can verify this:
```
docker exec nginx ip addr
```
We can now run our busybox container with --network ngnet to be on this network:
```
docker run --rm -t -i --network ngnet busybox wget http://nginx/
```
which should return something like:
```
Connecting to nginx (172.X.X.X:80)
wget: server returned error: HTTP/1.1 403 Forbidden
```
Then, you can try:
```
docker run --rm -t -i --network ngnet busybox wget http://nginx/python3/index.html
```

Part 5. Using the builder

The objective of this part is to build and run 4 services:

nginx: a simple HTTP server that serves static content
etherpad: a collaborative editing application that will require persistent storage in an external volume
etherpad-mysql: the same etherpad application but configured to use a mysql database as storage backend (and running in a separate container)
mini-httpd: a webserver that we want to patch, this will require a multi-stage build

These examples are provided with a docker compose configuration (docker-compose.yml) to ease their deployment. The documentation for docker compose is available at https://docs.docker.com/compose/.

Here are the main commands:

docker compose up to deploy all the containers, if not present in the engine, they will be automatically pulled from the registry (for containers using the "image:" option) or built from source (for those using the "build:" option)
docker compose build to force rebuilding the images (useful when the sources were modified)
docker compose down to clean up everything
docker compose run CONTAINER [ARG0 ARG1 ...] to run a container with the same config as the container CONTAINER defined in docker-compose.yml (useful for debugging when nothing works -> just run docker compose run CONTAINER bash to have a shell)

By default docker compose up works in the foreground. If the command is interrupted (Control-C), all containers are stopped and the command terminates. You can use docker compose up -d to launch it in the background.

nginx server

We will build an image with an nginx server from the debian distribution and run it to serve some static content.

This example is located in the nginx directory:

nginx/nginx/ -> contains sources of our docker image (Dockerfile)
nginx/www/ -> contains the static files we want to serve
nginx/log/ -> will store the server logs
nginx/docker-compose.yml -> docker compose configuration

Procedure:

Go to the nginx/ directory and have a look to the docker compose configuration
Run docker compose up to build the image and run the container. Note: the build should work but the container will stop immediately (because nginx is not yet installed).
Edit the Dockerfile to have the nginx package installed in the image and override the default container command so that nginx is run when we do docker compose up. Note: by default docker compose up will not rebuild the image, you should run docker compose up --build (or run docker compose build before).
Once it is running, open http://localhost:8080/ in your browser to see it working and check the log/ directory to ensure that the logs are there.

Note: on Windows/MacOS, if your docker engine is running inside a virtual machine, then you need to configure a port redirection in VirtualBox (i.e redirect host TCP connections to 127.0.0.1:8080 towards the 8080 port of the guest)
In your Dockerfile, you may add an EXPOSE instruction to declare that the container listens to its port 80 and VOLUME instructions to tell that /var/www/html and /var/log/nginx are expected to be mounted from external volumes.

etherpad (dirtydb backend)

In this part, we will run a real web application (etherpad) and have its persistent data stored in an external volume. We will use etherpad's default backend (dirty db).

This example is located in the etherpad directory:

etherpad/etherpad/Dockerfile -> Dockerfile draft
etherpad/etherpad/1.6.1.zip -> sources of the 1.6.1 release
etherpad/var/ -> external volume
etherpad/docker-compose.yml -> docker compose configuration

The installation procedure is documented here: https://github.com/ether/etherpad-lite/tree/1.6.1#installation. You should keep it open, and refer to it during the exercise because we do not give all the details! We will use the default config and install the sources from the 1.6.1.zip archive provided in this directory.

Note: the Dockerfile draft is based on Debian 10 "buster", which reached its End-of-Life and no longer receives security updates. We use it on purpose: etherpad 1.6.1 does not run on the recent nodejs versions shipped by newer Debian releases, and keeping this old setup lets you observe a behaviour of the application that has been fixed in modern releases (see the graceful stop step).

Since the regular Debian mirrors no longer serve buster, the draft already configures apt to download the packages from the archive (archive.debian.org). This is fine for this exercise, but do not use an End-of-Life base image for real applications.

Cd to etherpad/ and have a look to the docker compose configuration and to the draft Dockerfile
Modify the Dockerfile to install the required debian packages. Then run docker compose build.
Continue to modify the Dockerfile to:
- extract the sources in /opt/etherpad/ and make them owned by the etherpad user
- set the default runtime configuration
  - run as user etherpad
  - run the command bin/run.sh
  - run inside the directory /opt/etherpad/
(first run) Once you are done, run:
```
docker compose up
```
Once etherpad is started, open http://localhost:9001/ in your web browser and test the app.

Note: on Windows/MacOS, if your docker engine is running inside a virtual machine, then you need to configure a port redirection in VirtualBox (i.e redirect host TCP connections to 127.0.0.1:9001 towards the 9001 port of the guest)
(persistence) Now we will ensure that the data are effectively stored in the external volume and that we still have them when we restart the container. Once the app is working, create a pad and put some text in it.

Then Stop the container (hit Ctrl-C) and remove it completely with:
```
docker compose rm etherpad
```
List the content of the var/ subdirectory, normally the dirtydb files should be there.

Relaunch the container and reopen the pad in your browser. It should still be there.
(startup time) When running etherpad, you noticed that its startup script installs some npm dependencies before running the app. This normally happens only during the first run. Unfortunately since we are running in an immutable image (the container filesystem is copy-on-write) all the packages are dropped when the container is removed and they thus have to be reinstalled at every subsequent run.

Inside bin/run.sh there is a line that runs bin/installDeps.sh before running etherpad. This is the script that installs the dependencies.

Modify your Dockerfile so that bin/installDeps.sh is executed at build time, then rebuild and relaunch the container twice.
```
docker compose build
docker compose up
(Ctrl-C)
docker compose rm
docker compose up
```
The second run should be much faster.
(graceful stop) Stop your container and restart it in the background.
```
docker compose stop
docker compose up -d
```
Now stop it:
```
docker compose stop
```
The container should have stopped quickly. If it takes 10 seconds, then this means that the container did not stop cleanly, because docker stop forces the container to stop after 10 seconds with a SIGKILL signal. This is bad because we do not want to play with the devil and corrupt our data.

Check the output of the container:
```
docker compose logs
```
There is effectively no messages of etherpad shutting down. It did not catch the termination signal from the docker engine.

Restart the container, display the list of processes inside the container.
```
docker compose up -d
(wait until ready)
docker top etherpad-etherpad-1
```
There should be only one process: the nodejs server. If you have another bash (or sh) process above it, then it means that node is a child process of bash. When docker stops a container, it sends the termination signal (SIGTERM) to the root process only. Because the shell script does not forward this signal, the node server cannot know it has to stop.

Look at bin/run.sh and find the place where nodejs is run.
```
docker compose run etherpad cat bin/run.sh | less
```
Its actually the last line and the command is prefixed with exec for replacing the current process (which is right), therefore run.sh is not the culprit.

If it is not, then the culprit is the Dockerfile. Check your CMD instruction. It may look like one of:
- CMD ["bin/run.sh"]
- CMD bin/run.sh
The first form is ok, it tells docker to execute ["bin/run.sh"]. The second is not. When provides as a string, the CMD command is run in a shell. Thus the second form is equivalent to CMD ["sh", "-c", "bin/run.sh"].

If in this case, fix your CMD line, rebuild the image, start the container, check with docker top that sh is gone and stop with docker compose stop.

...

The container does still not stop properly!

We have actually a second issue (or a first issue, if you did the CMD line right), but there is a hint: when we started the container in the foreground (docker compose up), it stopped gracefully when hitting Ctrl-C. In fact nodejs expects to be terminated with the SIGINT signal, rather than with SIGTERM (the default signal sent by docker and by the Unix kill command).

Add a STOPSIGNAL line to your Dockerfile to tell docker that this container must be stopped with SIGINT. Then rebuild and test your image, the container should now stop properly.
(healthcheck) When running in production, if our container crashed and the process terminates, docker can report it to us (the container has terminated!). However, if it freezes, the docker engine will have no clue that something wrong is happening inside the container.

We can provide a HEALTHCHECK command in our Dockerfile. When present, docker will run the command periodically when the container is up and display its result in the status of the container.

Add a HEALTHCHECK command that ensures that the app is responding (you can use the curl utility for that). Then rebuild and restart, the healthcheck status should appear in docker ps.
(session key) Our container is running and our data are in the var/ external volume. We want to make sure we did not miss anything, we will check that with docker diff.
```
docker compose up -d
(wait until ready)
docker diff etherpad-etherpad-1
```
This command shows the differences between the container filesystem and the docker image used to run it. In the list we can see that the session key SESSIONKEY.txt is not stored in the external volume, which implies that it will be dropped across restarts.

While this is not critical for the app (a new key is regenerated anyway), it can be a burden for the users because their sessions will be closed when the server is restarted.

Modify the Dockerfile to have the session key stored in the var volume (for example with a symbolic link).
(metadata) You should add an EXPOSE and a VOLUME line to your Dockerfile to inform about the listened tcp port and the external volume.

etherpad (mysql backend)

As a second step, we will extend our etherpad image to make it use a mysql server as a storage backend.

This exercise is located in the etherpad-mysql directory. The Dockerfile starts with FROM etherpad-etherpad to use the image from the previous exercise as a base. For the mysql container we will use the official image provided by the mariadb project.

You will need to:

configure the mysql container to provide a database and a user account for etherpad. The documentation of the mariadb image is available at https://hub.docker.com/_/mariadb. You may have to tune some environment variables for the mysql container in your docker-compose.yml.
provide a settings.json file in your etherpad image to use the mysql database.
once done, run docker compose up and see what happens.
if you have extra time, you can modify the etherpad image to delay the execution of nodejs until the mysql server is ready and responding.

NB: The etherpad container can reach the database using the hostname mysql (the name of the service), since docker compose connects all the containers of the project to the same network.

mini-httpd

In this exercise, we will build mini-httpd from sources and install it in a docker image. This implies a multi-stage build, as your Dockerfile will build two images:

a first image with the development toolchain (compiler,...) that will build the binary
a second image which will be the final image, it just needs to contain the binary we built in stage 1 + the minimal runtime dependencies.

Once it is working, you can modify the first stage to patch the sources of mini-httpd (for example to change the background color of the directory indexes).

Navigation Menu

Repository navigation

About

Repository files navigation

Docker Exercises

Part 2. Containers

Part 3. Container inputs/outputs

Part 5. Using the builder

nginx server

etherpad (dirtydb backend)

etherpad (mysql backend)

mini-httpd

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
etherpad-mysql		etherpad-mysql
etherpad		etherpad
mini-httpd		mini-httpd
nginx		nginx
.gitignore		.gitignore
README.md		README.md

Latest commit

History

Folders and files

Repository files navigation

Docker Exercises

Part 2. Containers

Part 3. Container inputs/outputs

Part 5. Using the builder

nginx server

etherpad (dirtydb backend)

etherpad (mysql backend)

mini-httpd

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages