Running Docker Containers (from AWS ECR) with Systemd

Systemd units can be used to automatically start Docker containers on boot. Systemd, by default, will also monitor for the container dying and restart it if this happens.

Unit File

The unit file is fairly simple though there’s a few small nuances if you’re using AWS ECR.

Create a unit file at /etc/systemd/system/docker.your-container.service. Syntax is as follows:

[Unit]
Description=Your Container Name
After=docker.service
Requires=docker.service
StartLimitInterval=200
StartLimitBurst=10

[Service]
TimeoutStartSec=0
Restart=always
RestartSec=2
ExecStartPre=-/usr/bin/docker exec %n stop
ExecStartPre=-/usr/bin/docker rm %n
ExecStartPre=/usr/bin/bash -c 'docker login -u AWS -p $(aws ecr get-login-password --region eu-west-1) 0123456789.dkr.ecr.eu-west-1.amazonaws.com'
ExecStartPre=/usr/bin/docker pull 0123456789.dkr.ecr.eu-west-1.amazonaws.com/your-container:latest
ExecStart=/usr/bin/docker run --rm --name %n 0123456789.dkr.ecr.eu-west-1.amazonaws.com/your-container:latest

[Install]
WantedBy=multi-user.target

There’s a few parts to edit here:

• 0123456789 is a dummy AWS account ID. Replace this with your own
• eu-west-1 needs replacing with your preferred region, if it differs
• your-container:latest is the name and tag of your container, in ECR

A few key concepts are as follows:

• The naming of the unit file doesn’t really matter, though %n is replaced with this. When your container runs, it is given a name which matches the unit file
• The unit file will attempt 10 restarts, 2 seconds apart. This is important as a race condition might exist between a container dying and systemd restarting it

Running the Container

Once the unit file is in place, reload systemd:

systemctl daemon-reload

Now you can start the container and check it is running:

systemctl start docker.your-container.service
docker ps

If you stop the docker container, systemd will restart it for you within a couple of seconds:

docker stop docker.your-container.service
sleep 4
docker ps

If you actually want to stop it, use systemd:

systemctl stop docker.your-container.service

Container won’t stop with systemd

This is a bit of a gotcha that had me scratching my head for a while. The systemctl stop command took ages to run and then finished without stopping the container. systemctl stop will send a SIGTERM to Docker. Docker will send the SIGTERM to your main process (pid 1). If the main process doesn’t handle this SIGTERM then the container won’t stop. You’ll also notice that docker stop takes 10 seconds or so; this is because it waits 10 seconds and then sends a SIGKILL to forcefully kill the process.

In my case, my Dockerfile CMD[] was a bash script. The script did some stuff and then ran sleep 60 inside an infinite loop. Bash will wait for the sleep to finish, before it handles the SIGTERM. The solution here was to reduce the sleep time to 1 second (i.e. sleep 1).

Was this Post Helpful?

Then buy me a beer (please)!