Remote Docker

Your Docker client (like the docker command) can be on a separate computer than the docker daemon. If you ever played with docker machine (depreciated), this is basically that idea.

Docker clients talk to Docker daemons through an API.

Example clients are the usual suspects that Docker users are familiar with: the docker and docker-compose commands (among others).

Since the Docker daemon just exposes an API, we can connect to it remotely! This is a feature we use at Chipper CI.

You can easily point your docker command to a remote host by setting a DOCKER_HOST environment variable:

# List running containers on host 192.168.23.54
DOCKER_HOST=192.168.23.54:2375 docker ps

This of course assumes you can access that host, the daemon is listening on appropriate networks, and no network security is blocking that port from accepting external connections.

Remote Docker in Chipper

Chipper allows the use of Docker within builds by providing remote servers that have Docker installed on them.

This is for security, since builds are not run on dedicated servers (a feature we're looking at for the future). If you ran docker ps and saw other's customer data…well, that just wouldn't do.

So, each build that requests docker as a service gets a server assigned to that build.

⚠️ Docker is a paid feature in Chipper CI

version: 1

environment:
  php: 8.1
  node: 16

# Let us know you want Docker in this build
services:
  - docker:

pipeline:
  - name: Setup
    cmd: echo "composer install, or something"

Under the Hood at Chipper

Under the hood, we have a auto-scaling group (ASG) with (you guessed it) servers. These servers have Docker on them, and are in a private network that the Chipper build servers can reach.

As a server is requested for a build, a server is "popped" out of the ASG, and used in that build. That Docker server is destroyed when the build is complete. The ASG, in the meantime, replaces the server taken out of the ASG. A new one becomes ready for another build.

A bit more interesting is that there's also a "warm-pool" of servers in that ASG. These are servers that have been created, bootstrapped, but then turned off. If extra servers are needed, these are spun up and used in a similar way. "Warm" servers have faster boot-times than "fresh" servers, and so extra demand for Docker servers are able to be met fairly quickly this way.

Using Docker Remotely

There are some caveats to run Docker remotely (whether in Chipper or outside of it). Docker is literally running on a separate server, so you can imagine some things can get weird - for example, file sharing.

Docker build works!

First, the good news. Running docker build works great. The COPY and ADD keywords will be able to copy your local code files into a Docker image when running docker build commands.

This lets the vapor deploy command work (if they use the Docker runtime), since Vapor is able to build files into the resulting Docker image.

You can also build your own custom images, if that's part of your CI process.

No localhost

More good news (or at least, not bad news). You can still use Docker to provide some extra services that you may want in your CI pipeline.

Normally you'd expose ports for a given Docker service and use [localhost](http://localhost) to call them:

docker run -p 1337:1337 my-image:latest

# nope
curl localhost:1337/v1/some/api

In our case, Docker is not run locally, so [localhost](http://localhost) won't work. However, we know the address of the Docker server, and we can use that!

docker run -p 1337:1337 my-image:latest

# yes!
curl $DOCKER_HOST_IP:1337/v1/some/api

Again, this assumes the host allows connections on port 1337 from wherever you are making a curl request to.

No file system sharing

Mounting directories into your Docker container simply won't work. There's no magic networking configuration (on Chipper's end) to connect 2 different server's file system. In theory that's possible, but it's a relatively strange setup to find in the wild.

That means the following won't work:

# nope
docker run \
    -v $(pwd):/var/www/html \
    -p 80:80 some-image:latest

As a result, many docker-compose configurations (which often share local volumes into a container) are likely to fail in Chipper CI.

If you need to run a container that sees your code files, you'll need to get files into the container via docker build, and then run the resulting image as a new container.

More information on using Docker in Chipper CI is available here, including a list of environment variables we setup for you when using Docker.