Using CircleCI for Continuous Integration of Elixir projects

Continuous integration is vital to ensure the quality of any software you ship. Circle CI is a great tool that provides effective continuous integration. In this blog, we will guide you through how to set up CircleCI for an Elixir project. We’ve set up the project on GitHub so you can follow along. In the project we will:

• Build the project
• Run tests and check code coverage
• Generate API documentation
• Check formatting
• Run Dialyzer check.

You can follow the actual example here. Some inspiration for this blog post comes from this blog and this post. Prerequisites: For this project you will need an account on CircleCI and GitHub. You also need to connect the two accounts. Lastly, you will need Erlang/OTP and Elixir installed.

Create an Elixir project

To begin with, we need a project, for this demonstration we wanted to use the most trivial possible example, so we generated a classic ‘Hello World’ program.
To create the project, simply type the following into the shell:

mix new hello_world

We also added a printout, because the generated constant-returning function can be optimised away and that will confuse the code coverage checking.

Add code coverage metric

Code coverage allows us to identify how much of the code is being executed during the testing. Once we know that, we can also understand what lines are not executed because these lines could be where bugs can hide undetected or that we have forgotten to write tests for. If those lines of code are unreachable or not used, they should obviously be removed. To get this metric, I add the excoveralls package to our project (see here for details). Beware that even if you have 100% code coverage, it does not mean the code is bug-free. Here’s an example (inspired by this):

defmodule Fact do

  def fact(1), do: 1
  def fact(n), do: n*fact(n-1)
end

If the test executes Fact.fact(10), we get 100% test coverage, but the code is still faulty, it won’t terminate if the input is e.g. -1. For a new project, it should be easy to keep the code coverage near 100%, especially if the developers follow test-driven principles. However, for a “legacy” or already established project without adequate test coverage, reaching 100% code coverage might be unreasonably expensive. In this case, we should still strive to increase (or at least not decrease) the code coverage.

To run the tests with code coverage, execute:

mix coveralls.html

In the project root directory (hello_world). Apart from the printout, a HTML report will be generated in the cover directory too.

Add Dialyzer checking

Dialyzer is a static analyzer tool for Erlang and Elixir code. It can detect type errors (e.g. when a function expects a list to be an argument, but is called with a tuple), unreachable code and other kinds of bugs. Although Elixir is a dynamically typed language, it is possible to add type specifications (specs) for function arguments, return values, structs, etc. The Elixir compiler does not check for this, but the generated API documentation uses the information and it is extremely useful for users of your code. As a post-compilation step , you can run Dialyzer to check the type specifications in addition to writing unit-tests, you can regard this type checking as an early warning to detect incorrect input before you start debugging a unit-test or API client. To enable a Dialyzer check for this code, I’ll use the Dialyzer mix task (see this commit for more details). Dialyzer needs PLT (persistent lookup table) files to speed up its analysis. Generating this file for the Elixir and Erlang standard libraries takes time, even on fast hardware, so it is vital to reuse the PLT files between CI jobs.

To run the dialyzer check, execute ‘mix dialyzer’ in the project root directory (hello_world). The output will be printed on the console. The first run (which generates the PLT file for the system and dependencies) might take a long time!

Add documentation generation

Elixir provides support for written documentation. By default this documentation will be included in the generated beam files and accessible from the interactive Elixir shell (iex). However, if the project provides an API for other projects, it is vital to generate more accessible documentation. We’re going to use the mix docs task to generate HTML documentation (see this commit for more details).

To generate documentation, execute:

‘mix docs’

In the project root directory (hello_world). The documentation is generated (by default) in the docs directory.

Ensure code formatting guidelines

Using the same code style consistently throughout a project helps readers understand the code and could prevent some bugs from being introduced. The mix tool in Elixir provides a task to check that the code is in compliance with the specified guidelines. The command to do this is:

mix format --check-formatted

In the project root directory (hello_world).

Push the project to GitHub

Now that the project is created, it needs to be published to allow CircleCI to access it. Create a new repository on GitHub by clicking New on the Repository page, then follow the instructions. When the new repository is created, initialize the git repository and push it to GitHub:

cd hello_world
git init
git add config/ .formatter.exs .gitignore lib/ mix.exs README.md test/
git commit -m “Initial version” -a
git remote add origin <repo-you-created>
git push -u origin master

Integrate the GitHub repository to CircleCI

Login to circleci.com Click on “Add projects” Click on “Set Up Project” for hello_world For now skip the instructions to create the .circleci/config.yml file (we’ll get back to this file later), just click on “Start Building”

The build will fail, because we didn’t add the configuration file, that’s the next step.

Configuring the CircleCI workflow

As our requirement states above, we’ll need 5 jobs. These are:

• Build: Compile the project and create the PLT file for Dialyzer analysis.
• Test: Run the tests and compute code coverage.
• Generate documentation: Generate the HTML and associated files that document the project.
• Check code format: The mix tool can be used to ensure that the project follows the specified code formatting guidelines.
• Execute Dialyzer check: run the Dialyzer mix task using the previously generated PLT file

The syntax of the CircleCI configuration file is described here.

The next section describes the configuration required to setup the above five jobs, so create a .circleci/config.yml file and add the following to it:

Common preamble

version: 2.1
jobs:

The above specifies the current CircleCI version. The jobs (described in the next sections) should be listed in the configuration file.

The build step

The configuration for the build step:

  build:
    docker:
    - image: circleci/elixir:1.8.2
        environment:
        MIX_ENV: test

    steps:
    - checkout

    - run: mix local.hex --force
    - run: mix local.rebar --force

    - restore_cache:
        key: deps-cache-{{ checksum "mix.lock" }}
    - run: mix do deps.get, deps.compile
    - save_cache:
        key: deps-cache-{{ checksum "mix.lock" }}
        paths:
            - deps
            - ~/.mix
            - _build

    - run: mix compile

    - run: echo "$OTP_VERSION $ELIXIR_VERSION" > .version_file
    - restore_cache:
        keys:
            - plt-cache-{{ checksum ".version_file" }}-{{ checksum "mix.lock" }}
    - run: mix dialyzer --plt
    - save_cache:
        key: plt-cache-{{ checksum ".version_file"  }}-{{ checksum "mix.lock" }}
        paths:
            - _build
            - deps
            - ~/.mix

In this example, I’m using 1.8.2 Elixir. The list of available images can be found here. The MIX_ENV variable is set to test, so the test code is also built and more importantly, the PLT file for Dialyzer will be built for test-only dependencies too.

Further build steps

The actual build process checks the code, fetches and installs hex and rebar locally.

Restore the dependencies from cache. The cache key depends on the checksum of the mix.lock file which contains the exact versions of the dependencies, so this key changes only when actual dependencies are changed. The dependencies and built files are saved to the cache, to be reused in the test and documentation generating steps, and later when CI runs.

Build the actual project. Unfortunately, this result cannot be reused, because the checkout steps produce source files with current timestamp, so in later steps, the source files will have newer timestamps than the beam files generated in this step, this will lead to mix compiling the project anyway.

The dialyzer PLT file depends on the Erlang/OTP and Elixir versions. Even though I’ve fixed the Elixir version, it is possible that the Erlang/OTP in the Docker image is updated and in that case, the PLT file would be out of date. As the CircleCI caches are immutable, there’s no way to update the PLT file, for these cases, you’ll need a new cache name for new cache contents. Unfortunately, not all environment variable names can be used in CircleCI cache names, so I needed to use a workaround here: create a temporary .version_file which contains the Erlang/OTP and Elixir versions and use its checksum in the cache name along with the checksum of the mix.lock file (which contains the exact versions of all dependencies). So as long as we have the exact same dependencies and versions, we can reuse the PLT file safely, but as soon as anything changes, we get to use a new PLT file.

The test step

The configuration for the test step:

 test:
    docker:
    - image: circleci/elixir:1.8.2

    steps:
    - checkout
    - restore_cache:
        key: deps-cache-{{ checksum "mix.lock" }}
    - run: mix coveralls.html

    - store_artifacts:
        path: cover
        destination: coverage_results

Obviously, you need to use the same docker image as you use in the build step. There’s no need to explicitly configure the MIX_ENV environment variable because the mix job will set it. The test is fairly straightforward: checkout the code from the repository, fetch the dependencies from the cache, then run the coverage check job. The coverage report is generated in the cover directory. It is stored as an artifact, so you can check the results via a browser on the CircleCI page. If the tests itself fail, the output of the step will show the error message, and the job itself will fail.

The documentation generation step

The configuration for the documentation generation step:

    generate_documentation:
    docker:
    - image: circleci/elixir:1.8.2
        environment:
        MIX_ENV: test

    steps:
    - checkout
    - restore_cache:
        key: deps-cache-{{ checksum "mix.lock" }}
    - run: mix docs

    - store_artifacts:
        path: doc
        destination: documentation

Once again, the same docker from the build step needs to be used and setting up the MIX_ENV variable is important, otherwise, the dependencies might be different from the dev environment to the test environment. The documentation generation is fairly straightforward: checkout the code from the repository, fetch the dependencies from the cache (which contains the documentation generating task), then run the documentation generating the job. The documentation is generated in the doc directory. It is stored as an artifact, so you can check the results via a browser on the CircleCI page.

The dialyzer step

The configuration for the dialyzer step:

  dialyzer:
    docker:
    - image: circleci/elixir:1.8.2
        environment:
        MIX_ENV: test

    steps:
    - checkout
    - run: echo "$OTP_VERSION $ELIXIR_VERSION" > .version_file
    - restore_cache:
        keys:
            - plt-cache-{{ checksum ".version_file" }}-{{ checksum "mix.lock" }}
    - run: mix dialyzer --halt-exit-status

Much like the last step, the docker image needs to match the one used for the build. Ensure that the MIX_ENV variable is correct. The workaround mentioned above is required to find the cache with the right PLT file, then executing Dialyzer is a simple command. If Dialyzer finds an error, it will return with a non-zero exit code, so the step will fail.

The format-checking step

The configuration for the format checking step:

  format_check:
    docker:
    - image: circleci/elixir:1.8.2
        environment:
        MIX_ENV: test

    steps:
    - checkout

    - run: mix format --check-formatted

This is really simple, we don’t even need any cached files, just run the check.

Piecing it all together

CircleCI executes a workflow which contains the above steps. This is the configuration for the workflow:

workflows:
  version: 2
  build_and_test:
    jobs:
    - build
    - format_check:
        requires:
            - build
    - generate_documentation:
        requires:
            - build
    - dialyzer:
        requires:
            - build
    - test:
        requires:
            - build

The workflow specifies that the four later steps depend on the build test, but they can be executed simultaneously. The whole configuration file can be seen on GitHub. When the configuration file is ready, add it to git:

git add .circleci/config.yml

And push it to GitHub:

git push origin master

CircleCI will automatically detect that a new version was committed and will execute the configured jobs.

Conclusion

Continuous integration is essential in a fast-moving project. The developers need feedback as soon as possible because the earlier a bug is found, the earlier and cheaper it is to fix it. This blog presents a simple, but effective setup that can run this vital part of an Elixir project. Want more fantastic Elixir inspiration? Don’t miss out on Code Elixir, a day to learn, share, connect with and be inspired by the Elixir community. Want to learn about live tracing in Elixir? Head to our easy-to-follow guide.