Your Docker client (like the
docker command) can be on a separate computer than the docker daemon. If you ever played with
docker machine (depreciated), this is basically that idea.
Docker clients talk to Docker daemons through an API.
Example clients are the usual suspects that Docker users are familiar with: the
docker-composecommands (among others).
Since the Docker daemon just exposes an API, we can connect to it remotely! This is a feature we use at Chipper CI.
You can easily point your
docker command to a remote host by setting a
DOCKER_HOST environment variable:
# List running containers on host 192.168.23.54 DOCKER_HOST=192.168.23.54:2375 docker ps
This of course assumes you can access that host, the daemon is listening on appropriate networks, and no network security is blocking that port from accepting external connections.
Remote Docker in Chipper
Chipper allows the use of Docker within builds by providing remote servers that have Docker installed on them.
This is for security, since builds are not run on dedicated servers (a feature we're looking at for the future). If you ran
docker ps and saw other's customer data…well, that just wouldn't do.
So, each build that requests
docker as a service gets a server assigned to that build.
⚠️ Docker is a paid feature in Chipper CI
version: 1 environment: php: 8.1 node: 16 # Let us know you want Docker in this build services: - docker: pipeline: - name: Setup cmd: echo "composer install, or something"
Under the Hood at Chipper
Under the hood, we have a auto-scaling group (ASG) with (you guessed it) servers. These servers have Docker on them, and are in a private network that the Chipper build servers can reach.
As a server is requested for a build, a server is "popped" out of the ASG, and used in that build. That Docker server is destroyed when the build is complete. The ASG, in the meantime, replaces the server taken out of the ASG. A new one becomes ready for another build.
A bit more interesting is that there's also a "warm-pool" of servers in that ASG. These are servers that have been created, bootstrapped, but then turned off. If extra servers are needed, these are spun up and used in a similar way. "Warm" servers have faster boot-times than "fresh" servers, and so extra demand for Docker servers are able to be met fairly quickly this way.
Using Docker Remotely
There are some caveats to run Docker remotely (whether in Chipper or outside of it). Docker is literally running on a separate server, so you can imagine some things can get weird - for example, file sharing.
Docker build works!
First, the good news. Running
docker build works great. The
ADD keywords will be able to copy your local code files into a Docker image when running
docker build commands.
This lets the
vapor deploy command work (if they use the Docker runtime), since Vapor is able to build files into the resulting Docker image.
You can also build your own custom images, if that's part of your CI process.
More good news (or at least, not bad news). You can still use Docker to provide some extra services that you may want in your CI pipeline.
Normally you'd expose ports for a given Docker service and use
[localhost](http://localhost) to call them:
docker run -p 1337:1337 my-image:latest # nope curl localhost:1337/v1/some/api
In our case, Docker is not run locally, so
[localhost](http://localhost) won't work. However, we know the address of the Docker server, and we can use that!
docker run -p 1337:1337 my-image:latest # yes! curl $DOCKER_HOST_IP:1337/v1/some/api
Again, this assumes the host allows connections on port
1337 from wherever you are making a
curl request to.
No file system sharing
Mounting directories into your Docker container simply won't work. There's no magic networking configuration (on Chipper's end) to connect 2 different server's file system. In theory that's possible, but it's a relatively strange setup to find in the wild.
That means the following won't work:
# nope docker run \ -v $(pwd):/var/www/html \ -p 80:80 some-image:latest
As a result, many
docker-compose configurations (which often share local volumes into a container) are likely to fail in Chipper CI.
If you need to run a container that sees your code files, you'll need to get files into the container via
docker build, and then run the resulting image as a new container.
More information on using Docker in Chipper CI is available here, including a list of environment variables we setup for you when using Docker.