Parallelized Coverage Tests with CircleCi and CodeClimate

When you are trying to hit a minimum of 90% test coverage, as we do here at FullStack Labs, you're bound to run into some lengthy test runs in your CircleCI Jobs. Optimizing your tests with parallelism and workspaces is achievable in a few key steps. First, build your project and persist your mounted directory, then run your tests using parallel jobs, and finally persist any artifacts you may want from the output.

Build & Test

To build and persist your application to the workspace, you will use persist_to_workspace. Below is an example that builds a typical Node.js application.

build:
    steps:
      - run:
          name: Install Node Modules
          command: npm install
      - persist_to_workspace:
          root: ~/repo
          paths:
            - ./*

NOTE: In this example, we persist all of the data in the repo directory. This is our working_directory. You can choose to persist individual files using the paths configuration.

In your test job, you need to set the parallelism flag to the max number of parallel jobs you would like to have. Then, attach your workspace using attach_workspace and run your tests.

test:
    parallelism: 4
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Run Tests
          command: npm run test:ci

Next, configure your tests by figuring out your best use case when splitting files. In one of our projects, we chose to create our own partitions using CIRCLE_NODE_INDEX and CIRCLE_NODE_TOTAL as FusionJS and our custom configuration did not play well with the slice of files.

After you’ve pushed your changes up you can see the separate jobs passing and failing as needed showing you their statuses and output.

As a bonus, you can split up your testing and lint jobs to have them fail separately without having to install and build any prerequisites since you can attach_workspace.

lint:
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Run Lint
          command: npm run lint

Collecting Coverage

Now that your tests are parallelized you will need to collect coverage using a similar method you selected for splitting your files. To do this, we need to install the cc-test-reporter and configure the builds. Adding the Install Test Reporter and Report Before Build steps to our build job. See Configuring Test Coverage on CodeClimate’s website for configuration and links to your specific build of the test reporter.

build:
    steps:
      - run:
          name: Install Test Reporter
          command: |
            wget -O ./cc-test-reporter https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64
            chmod +x ./cc-test-reporter
      - run:
          name: Report Before Build
          command: ./cc-test-reporter before-build
      - run:
          name: Install Node Modules
          command: npm install
      - persist_to_workspace:
          root: ~/repo
          paths:
            - ./*

Then we can add a Format Test Coverage step to our test job, including another persist_to_workspace for the tmp directory. We use the cc-test-reporter to format-coverage of our lcov.info output for this test partition.

NOTE: We are using the $CIRCLE_NODE_INDEX to define our coverage partitions. This may vary depending on how you decide to split your files as described above.

test:
    parallelism: 4
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Run Tests
          command: npm run test:ci
      - run:
          name: Format Test Coverage
          command: ./cc-test-reporter format-coverage -t lcov -o tmp/codeclimate.$CIRCLE_NODE_INDEX.json coverage/lcov.info
      - persist_to_workspace:
          root: ~/repo
          paths:
            - tmp

Next, we can add a reporting job that will sum our coverage and upload it to CodeClimate. Again, using the cc-test-reporter to sum-coverage of all of our test runs, then uploading them by defining the input JSON.

report:
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Upload Coverage To Code Climate
          command: |
            ./cc-test-reporter sum-coverage tmp/codeclimate.*.json -o tmp/codeclimate.total.json
            ./cc-test-reporter upload-coverage -i tmp/codeclimate.total.json

Finally, we can see all of the individual jobs on a pull request and the CodeClimate coverage results.

Here at FullStack Labs, we produce high-quality solutions for our clients that they love! If you like what I posted above and would like to join us, please visit our Careers page.