What is takes to do CI/CD for Laravel

There's a lot that goes into running CI/CD for a Laravel application!

Let's dig into the factors that go into successfully building, testing, and deploying Laravel. This will be about Chipper CI (duh), but the things we discuss here apply to CI/CD no matter where you do that.

We'll discuss the following:

1. The Build Environment
2. Environment Variables
3. Additional Services
4. Build Triggers
5. Build Pipeline
6. Notifications

The Build Environment:

You need to get your continuous integration environment setup with the correct dependencies for your application.

This means using the right version of Node and PHP. You may also require specific PHP modules or additional software.

The result is that you need to learn just enough about Docker, server stuff, or your clouds special snowflake setup to get your application running in CI.

Chipper CI makes this easy! Its build environment is just like Forge - setup for you with just about anything you need.

Similar to Forge, it runs an Ubuntu LTS version. This means the usual commands are available, such as apt-get to install additional software. You can even use sudo (without a password)!

Setting Up Your Environment

Setting up the environment is just a few lines in your project's .chipperci.yml file:

environment:
  php: 8.1
  node: 14

That's it! Since we're running Laravel, all we usually need to care about is the PHP and Node versions being used.

Under the hood, nvm is controlling the Node version, so you can set a specific Node version:

# Use a specific version of Node
nvm install 14.20.0
nvm use 14.20.0
nvm alias default 14.20.0

Other versions of PHP are installed as well. Setting the PHP version just sets the default version used for the php command. However, you can use any PHP version you need:

# Install composer dependencies with PHP 8.0
php8.0 /usr/bin/composer install

# Check that php7.4 is available
php7.4 -v

Environment Variables

The part that trips up the most people when setting up CI for Laravel is forgetting environment variables. It's very easy to forget about what env vars were set up, especially when you've been working on a project for months or years.

Your Laravel CI pipelines need the right environment variables! Where those environment variables go can vary.

Laravel uses the DotEnv package (a popular convention in many frameworks and languages), so it can read system environment variables, as well as variables defined in a .env file. Here's a few places you might setup environment variables in Chipper CI:

1. Project settings - These become “system” environment variables, available globally
2. Secrets - These also become “system” environment variables, but their values are hidden from build output
3. Create a .env.testing file in your code base

Chipper CI will set environment variables for you based on the service you select as well.

What Environment Variables to Set?

The simplest answer to this is “the least amount possible”.

By default, Chipper CI has only the following set for you:

# Make sure telescope isn't enabled, as
# it will slow down tests and...everything
TELESCOPE_ENABLED=false

You might be tempted to drop your entire local .env file into your CI environment. You certainly can if you want, but there's probably way more in there than you need.

Secrets

The first problem with that is related to secrets, such as API keys. Any sensitive items in your .env file should be secrets within Chipper CI. Additionally, you should only include secrets you actually need for continuous integration.

Unnecessary Variables

Chipper CI will actually remove certain environment variables (ones it will set itself). Of note are the APP_* ones! These are not saved in Chipper's environment settings, and so should be set in a .env or .env.testing file in your CI environment if you need them.

Encryption Key

The APP_KEY variable can (and should!) be set to a random (but valid) key value on each CI run to help prevent you from leaking encryption keys.

The APP_URL variable needs to be set for Dusk, but Dusk only reads from a .env or .env.dusk or .env.dusk.* file, and so that variable should be placed within those files only.

Minimal Variable Usage

A typical installation for me only uses ~3 environment variables! Something like this:

STRIPE_KEY=pk_test_xxxxx
TELEGRAM_BOT_USERNAME=chipperxxxx_bot
TELESCOPE_ENABLED=false

STRIPE_SECRET is set as a secret rather than an environment variable.

Then, within my Pipeline scripts, I run:

cp env.example .env
composer install
php artisan key:generate

Copying the .env.example file just ensures there's a file with APP_KEY= within it. Then the artisan key:generate command populates it on each CI run.

Environment Variable Precedence

It's important to understand environment variable precedence as well. Environment variable locations with the highest precedence will “win”, and will dictate what Laravel will see for a configuration value.

Here's the general overview, in order of most to least precedent when running tests:

1. PHPUnit <server> configuration defined in phpunit.xml
2. System environment variables (set in Project Settings / Secrets)
3. PHPUnit <env> configurations defined in phpunit.xml (not defined by default in Laravel projects)
4. .env.testing file
5. .env file

Note that your Project environment variables (and Secrets) can get over-written by the phpunit.xml file!

A caveat to this is Dusk testing. Dusk will look for a .env.dusk.* file, or use a .env file only. It will ignore system environment variables!

This is annoying because it means your Project Settings / Secrets need to get written to a .env (or .env.dusk) file before running browser tests.

Dusk also merges configuration files (.env and .env.dusk).

You can read more about Configuration Precedence in Laravel here.

Customized Configurations

If you've customized your application configurations, you may have a bit more work to do. One popular customization is changing the config/database.php configuration file. If your application no longer is looking for the standard DB_* variables, then you may need to set values for your custom environment variables within a .env.testing file in your repository.

For example, your .env.testing file may look like this, which can map the environment variables Chipper sets for you to your custom environment variables.

# File: .env.testing
#
# If your configuration uses custom environment variables
# one thing you can do is map the standard ones that Chipper sets
# to your custom ones:
CUSTOM_DB_CONNECTION="${DB_CONNECTION}"
CUSTOM_DB_USERNAME="${DB_USERNAME}"

Preparing for CI with Laravel

To help test your application for CI, I recommended starting a new copy (a fresh clone) of your project locally. Then follow the steps needed to make everything the same as you'd find in a CI environment.

This helps shake out what environment variables will be needed for a successful run in CI.

You can create a super minimal .env file, and (if needed) an empty, fresh database.

If done correctly, this method will be most like running Laravel in CI, where there is no real knowledge of the application beforehand.

Then you can run the same commands as you would in a CI pipeline. Installing composer dependencies, generating static assets, and running your tests.

Additional Services

Ideally you can run everything without external services (in-memory SQLite is amazing!), but real world applications require real world solutions.

If you need a database, Redis, or Docker, these are available to you in Chipper CI!

To enable a service, use the services section in the .chipperci.yml file.

services:
  # Choose only one of these databases
  - mysql: 5.7                      # 5.6, 5.7, 8
  - mariadb: 10.3
  - postgres: 13                    # 10, 11, 12, 13
  - postgis/postgis: 13-3.0         # 10-3.0, 11-3.0, 12-3.0, 13-3.0

  # Add redis, with the trailing colon
  - redis:

  # Paid accounts can add in Docker support,
  #  useful for running Vapor deployments.
  # Ensure you have the trailing colon.
  - docker:

In Chipper CI, services are configured for you. We set the Laravel-specific environment variables needed for you application to use to connect.

This means you do NOT need to set your DB_HOST, DB_USERNAME, REDIS_HOST, etc, environment variables. Chipper does that for you!

Dusk won't see these environment variables, so you'll need to build them into a .env.dusk file

Once you add a service to your Yaml file, they'll be part of your builds. Chipper CI ensures the services are running and ready for you before your pipeline commands are run.

Database Tips

One common need for mysql users is to give the default database user expanded permissions.

The first thing to know is that you can use the mysql command. It will run as user root and will not require a password.

mysql -e "grant all privileges on *.* to '$MYSQL_USER'@'%' with grant option;";

This command (especially the with grant option part) will give the MySQL user created expanded permissions to create databases. This is useful if you run tests in parallel or do other things that require creating databases on the fly.

Build Triggers

Chipper CI lets you define when a build is run. Builds can be triggered by commits, pull request events, or tags. The various Git providers (GitHub, GitLab, BitBucket) all send us webhooks when these events occur. Chipper CI parses them and determines if a build should be run.

For commits and pull request events, you can decide which branches (or branch patterns) will trigger a build. Builds will not be triggered for branches that don't match.

These are useful for having a setup such as “only build branches that will result in a deployment, or when certain pull requests are opened/updated”. That would look like this:

# Decide what events trigger a build
on:
  # "push" is a regular, old commit.
  # This will build on push to just 2 branches
  push:
    branches:
      - main
      - develop

  # This will build on PR open, re-open, and update
  # (Updates are when commits are pushed to a PR)
  # Here we say to only build when a PR is opened/reopened/updated 
  # and the branch being pushed to
  # is feature/.* or bug/.*
  pull_request:
    branches:
      - feature/.*
      - bug/.*

Omitting the on: section results in all builds being run, except for Pull Request events. For example, opening a pull request that does not also push a new commit will not result in a build.

Build Pipeline

The rubber meets the road! The build pipeline is where you define the commands to run in your CI builds.

These commands need to prepare your application, run tests, kick off deployments, and do anything else you may need.

Preparing your application comes dow to ensuring files and configuration that an application needs are in place. Often there's not much to do here - perhaps just ensuring an APP_KEY is set and then running composer install….

Here's Chipper's default pipeline:

pipeline:
  - name: Setup
    cmd: |
      cp -v .env.example .env
      composer install --no-interaction --prefer-dist --optimize-autoloader
      php artisan key:generate

  - name: Compile Dev Assets
    cmd: |
      npm ci --no-audit
      # Using Vite (use "npm run dev" for Mix):
      npm run build

  - name: Run Tests
    cmd: phpunit

We can see that the setup step has 3 commands. It copies the .env.example file, generating a .env file. It then runs composer install..., and finally runs artisan key:generate to ensure an APP_KEY is set (and is random!) for each CI run.

An alternative to this flow is to instead have a .env.testing file committed to your repository. It's completely up to you!

Note that the composer install... command will install dependencies as set by your composer.lock file. It's highly recommended that you commit a lock file to your git repository - it makes getting dependencies a lot faster, and should mirror what you have locally and in production. This helps makes your tests consistent with what you're about to deploy!

Using composer update instead will result in issues as you won't know what versions of your project's dependencies will get installed.

Other Setup Steps

Some times you may need to run additional setup steps. Perhaps you need to install some extra software! Commonly, some projects require that you migrate data into a test database ahead of running tests.

pipeline:

# ...

  - name: Setup Database for Testing
    cmd: |
      php artisan migrate --force
      php artisan db:seed

I recommend using SQLite's in-memory database for as long as you can get away with it. It makes everything faster!

However, many “real” applications start to use their databases specific features at one point or another, so Chipper CI offers the most popular databases for you.

Docker-Based Services

Those using Docker can add extra services as well. For example, if you need to run Meilisearch alongside your application, you can!

First, enable Docker (if your on a plan that includes the Docker feature):

services:
  - docker:

Then, within your pipeline, you can run a docker command to start a docker-based service.

pipeline:

  # ...

  - name: Start Meilisearch
    cmd: |
      docker run --rm -d -p 7700:7700 \
          -e MEILI_MASTER_KEY='SOME_MASTER_KEY' \
          getmeili/meilisearch:v0.28 \
          meilisearch --env="development"
      sleep 5

We run a new container instance of the official Meilisearch image. Then we sleep a few seconds to give it time to spin up.

After that, you'll be able to connect to Meilisearch over the network $DOCKER_HOST_IP:7700. Note that the $DOCKER_HOST_IP variable needs to be used to reach the instance, as it's running in a remote docker instance.

Static Assets

The default pipeline also installs your Node dependencies, and builds your static assets.

pipeline:

  # ...
  
  - name: Compile Dev Assets
    cmd: |
      npm ci --no-audit
      # Using Vite (use "npm run dev" for Mix):
      npm run build

This is done as most of us need the vite() or mix() helpers to not throw errors if there is no manifest file committed to your repository.

???? It's common to not have your manifest nor static assets committed to our repository when developing in teams, as it often results in pointless merge conflicts.

Of course, perhaps you don't need to run Node in your CI pipeline at all or, if you do, perhaps you can only build static assets if you need to. I recommend avoiding it if you don't need to - they are often the slowest part of any CI pipeline!

Testing

At some point you'll want to run your tests. This is usually done via phpunit, pest, or dusk.

How exactly you run your tests is up to you!

Chipper CI sets your $PATH to include ./compose/bin and ./node_modules/.bin (relative to the current directory you're in), so assuming you have phpunit as a dev project dependency, you should be able to simply run phpunit or pest. However you may also want to run php artisan test or php artisan duskdepending on your preferences and use case.

pipeline:

  # ...

  # Simple!  
  - name: Run Tests
    cmd: phpunit

Deploying

It is best practice to kick off deploys from your CI pipelines, as it ensures you are always continuously integrating (and delivering) your application. This builds up confidence both in your test suite and in your deployment practices.

Deploying Laravel applications can take many forms! Many of us use Forge or Envoyer.io, in which case deploying means sending a web hook. Others may use Vapor, in which case we run a vapor deploy command. There's also Fly.io, in which case you'll run a fly deploy command to kick off a deployment from your current code base (it will handle getting production dependencies and asset).

Finally, you can also grab the SSH key available per project and use it to grant SSH access to your web server. This way, your build pipeline an SSH into a server to run deployment commands!

Lastly, since the pipeline scripts are just bash, you can add conditionals! These are usually based on environment variables that Chipper CI sets per build. You can use this to decide what environment to deploy to!

Here's an example:

pipeline:

  # ...

  - name: Deploy
    cmd: |
      if [[ $CI_COMMIT_BRANCH == 'master' ]]; then
          # Maybe deploy to Vapor
          vapor deploy production
      fi

      if [[ $CI_COMMIT_BRANCH == 'develop' ]]; then
          # Maybe send a webhook to Forge or Envoyer
          curl -X POST https://my-webhook-url/develop/some-token \
              -d branch=$CI_COMMIT_BRANCH \
              -d commit=$CI_COMMIT_SHA
      fi

Notifications

Without notifications, you won't know the status of your project. This is important for failing tests and especially for failed deployments.

You can create notifications for Slack, Telegram, Discord, Microsoft Teams, and Email. These are found in Project Settings > Notifications.

Chipper CI build notifications have a bunch of options for how you are notified! You don't need to be flooded with notifications for each build.

My personal favorite is to receive notifications on build status changes (was passing but now is failing, and visa-versa).