Documentation

How it works

UndercoverCI is a robot who protects your Ruby codebase from untested code changes. It works with every Ruby project that can report test coverage with SimpleCov. Chances are, you're already doing that while running tests on a CI server.

The GitHub App reacts to commit and pull request events by queueing a new coverage check. Once your tests finish and upload the coverage report file, UndercoverCI parses the commit diff and correlates the structure of your changes (classes, modules, methods and blocks) against the uploaded test coverage report. The result is a passed or failed commit check reported to GitHub.

Passed

When the diff lines have full test coverage, you'll see a passed check with an additional breakdown of all changed code locations. Use it to find additional test coverage improvement opportunities in the surrounding lines and methods - reviewing your pull request is a good time to do that.

Failed

When untested lines or branches are found, a failed check will highlight them and ask you to add tests. Once tests are added, the comments will go away and the check status will turn to passed ✅.

---

Getting started

Follow these 3 steps to get started with UndercoverCI and receive automated test coverage comments.

1. Install the app on your GitHub account

Sign up from the home page or just follow the GitHub sign up link to sign up. You'll be prompted to install the app on your GitHub account or organization and give permissions to selected repositories.

Once installed, UndercoverCI will send checks for every commit and pull request. There is just one final step to receive meaningful coverage results in them.

2. Report test coverage data to UndercoverCI

UndercoverCI analyses each commit against a matching test coverage file. Make sure your Ruby project reports test coverage and uploads the report file to UndercoverCI by following the instructions below:

If your CI system isn't listed below, take a look at the general uploader documentation for more details.

3. Verify your setup

You have installed the UndercoverCI GitHub app and configured your CI to publish test coverage reports for analysis. In order to verify that your setup is working, create a branch and push a small code change to your repository – adding a dummy method should suffice to trigger a sample coverage warning:

If you made it here, it means your code review setup with UndercoverCI is ready!

Branch coverage

UndercoverCI and undercover CLI (starting with version >= 0.4.0) both accept coverage reports with branch coverage data.

The configuration example already shows how to enable branch coverage in SimpleCov, which is all you need to receive more fine-grained coverage checks.

If you decide to enable branch coverage reporting, every changed line of code with at least one uncovered branch will be flagged and result in a GitHub annotation for the entire method. This is a more rigorous approach compared to just validating line-by-line test coverage.

View coverage (beta)

Starting with undercover 0.9.0, UndercoverCI can also check ERB templates changed in a pull request. Your test suite needs Ruby 3.2+ and SimpleCov 1.0+. On older versions, pin undercover ~> 0.8.0 and keep checking Ruby code only.

1. Enable eval coverage in SimpleCov.start, next to branch coverage. Templates are only measured when this is on.

          SimpleCov.start do
            enable_coverage :branch, :eval
          end
        

If you merge reports from parallel tests, use the same enable_coverage :branch, :eval in the SimpleCov.collate block.

2. Opt in with a .undercover file in the repository root. -f replaces undercover's default include list and -x replaces the default exclude list, so list everything explicitly:

          -f *.rb,*.rake,*.ru,Rakefile,*.erb
        

If you need extra excludes, e.g. generator templates that tests never render, add them to the defaults:

          -x test/*,spec/*,db/*,config/*,*_test.rb,*_spec.rb,vendor/*,*/templates/*
        

if, unless, case, loops and do ... end blocks inside templates, including block helpers like <%= form_with ... do |f| %>, are checked alongside Ruby methods and reported on the template's own line numbers. If the uploaded report has no eval coverage at all, changed templates are skipped rather than flagged at 0%. Templates that can't be parsed are skipped as well.

Known limitations:

• Lines outside any if/case/loop/block aren't checked individually. A template that's never rendered and has no control flow isn't flagged.
• || and && fallbacks aren't measured by Ruby's coverage.
• With Rails 8.1, config.action_view.annotate_rendered_view_with_filenames = true shifts template line numbers by one. Turn it off in the test environment.
• ERB rendered outside Rails (stdlib ERB.new) needs a filename set.

View coverage is in beta; please report issues on GitHub.

Shorten the feedback loop with the local CLI

UndercoverCI coverage comments are generated by the undercover ruby gem that you can use locally too. While GitHub Check comments provide coding standards and consistency, you can shorten your feedback loop and check for missing coverage with every local commit thanks to the undercover CLI command. You can install it from RubyGems with:

          gem install undercover
        

Then, check for missing coverage inside uncommited changes by running specs and invoking undercover. Use the --compare flag to compare against a specific commit or branch, which is similar to what UndercoverCI performs when analysing your pushed pull requests:

          undercover --compare base-branch
          undercover --compare HEAD~1
        

Visit the undercover GitHub page to see more examples including integrations with Overcommit and Pronto hooks.

Set up your coding agent

If you're using AI coding agents like GitHub Copilot, Cursor, Claude Code or similar tools, you can guide them to write better tests with direct feedback from Undercover. The following system prompt example has been tested with Claude Code and helped it iterate towards correct coverage for the changes it made.

Note the following elements of the prompt:

• emphasize the importance of testing every code change
• document the local tooling well: how to run tests, lint and coverage
• document any other preferences about the desired output style to save more time

Example AGENTS.md configuration:

    ## Dev guidelines

    - All changes you make should be tested
    - Prefer updating existing tests (if they exist) over adding new test files
    - Always run tests and rubocop before completing your task
    - Do not add redundant comments
    - Do not say "You're absolutely right!"

    ## Bash commands

    bundle exec rake # run rspec tests and rubocop
    bundle exec rspec # run specs
    bundle exec rubocop # run rubocop lint
    bundle exec rubocop -A # correct correctable rubocop violations

    ## Test coverage

    bundle exec undercover # inspect coverage of local changes
    bundle exec undercover --compare master # inspect coverage w.r.t. master (for pull requests)
    bundle exec undercover --compare HEAD~1 # inspect coverage w.r.t. previous commit
        

Claude Code Plugin

undercover-claude is a Claude Code plugin that closes the undercover feedback loop directly inside your coding session. Read the experiment writeup to see how it compares to plain Claude Code runs.

Features:

• Sets up SimpleCov and undercover automatically at the start of a new project
• After every test run, checks which changed lines aren't covered and writes tests for the gaps
• Repeats until bundle exec undercover exits 0

Install:

git clone https://github.com/grodowski/undercover-claude
claude --plugin-dir ./undercover-claude

The plugin is pending Anthropic review. Once approved it will be installable with claude plugin install undercover.

Usage - run the coverage skill from any Claude Code session:

/undercover:coverage

---

Configuration

UndercoverCI uploader

The uploader script runs in your CI build environment and uploads coverage reports to UndercoverCI. This way the coverage files are processed against a git diff and generate a commit check. A sample call for the grodowski/undercover repository built on CircleCI could look like this:

          ruby -e "$(curl -s https://undercover-ci.com/uploader.rb)" -- \
                    --repo grodowski/undercover \
                    --commit $CIRCLE_SHA1 \
                    --simplecov coverage/coverage.json
        

The uploader.rb script is securely downloaded and evaluated with the required command-line options:

--repo

The $org/$repo formatted name matching exactly how your repository appears on GitHub. On GitHub Actions, a build running in a fork of that repository, such as a contributor's, uploads to the fork ($GITHUB_REPOSITORY) instead. When UndercoverCI isn't set up there, the uploader prints a notice and exits successfully, so forks of your repository don't get failing builds.

--commit

Current build commit SHA to identify on GitHub. This value will be provided by the CI build environment, e.g. $CIRCLE_SHA1 for CircleCI, ${{ github.event.pull_request.head.sha || github.sha }} for GitHub Actions or $TRAVIS_COMMIT for TravisCI. Consult your CI service documentation to get the right head SHA value for your build environment.

--pull-request

The pull request number, detected automatically on GitHub Actions pull request builds. Required to run coverage checks on pull requests opened from forks, because GitHub sends no check_suite event for those and the upload itself has to start the check. The value is optional, so the same build command works for push builds where it expands to nothing.

--pull-request-base-ref

The branch the pull request targets, sent with --pull-request so a fork's pull request is compared against the right branch. Defaults to $GITHUB_BASE_REF on GitHub Actions; elsewhere the default branch is used when it's not set.

--pull-request-head-ref

The pull request's branch, sent with --pull-request so the dashboard shows a fork's branch name. Defaults to $GITHUB_HEAD_REF on GitHub Actions.

--cancel

Cancel coverage check represented by the given --commit. The check will be passing and marked as completed + skipped on your GitHub commit or pull request.

--simplecov

A relative path to the SimpleCov JSON coverage report, defaults to coverage/coverage.json.

Parallel tests

If your CI build runs tests in parallel, there's an extra merge step to be performed before uploading coverage results, because the uploader script only accepts a single file at the time of writing this document. While partial coverage results might be supported in a future release, this Ruby snippet should get you started with merging multiple coverage reports:

          #!/usr/bin/env ruby
          # frozen_string_literal: true

          require 'simplecov'
          require 'undercover/simplecov_formatter'

          puts('Merging coverage results from parallel CircleCI tests containers into a single JSON report...')

          SimpleCov.collate(Dir['/home/circleci/rspec/*.resultset.json']) do
            enable_coverage :branch, :eval
          end

          report_path = ARGV[0] || 'coverage/coverage.json'
          SimpleCov.formatter = SimpleCov::Formatter::Undercover

          merged_result = SimpleCov.result
          merged_result.format!

          if File.size(report_path).zero?
            puts('Written report has 0 bytes')
            exit 1
          end
          puts("Done! Coverage saved to #{report_path}")
        

Parallel tests in Rails and Minitest

Try the snippet below to configure SimpleCov with parallelize in Rails and produce a single coverage output file that will be suitable for UndercoverCI. You can find more context in this GitHub issue.

          # test_helper.rb
          # ...
          parallelize(workers: :number_of_processors)

          parallelize_setup do |worker|
            SimpleCov.command_name "#{SimpleCov.command_name}-#{worker}"
          end

          parallelize_teardown do |worker|
            SimpleCov.result
          end
          # ...
        

Ignoring/skipping coverage with code comments

Pull requests in projects with low or nonexistent test coverage are likely to generate large numbers of check warnings. While the default workflow would be to address them before the PR approval, your strategy might be different.

In order to acknowledge an untested change and remove the UndercoverCI warning with the intention to improve later (or never), you can wrap the code block with SimpleCov's simplecov:disable and simplecov:enable comments, e.g.

          # simplecov:disable
          def skip_this_method
            never_reached
          end
          # simplecov:enable
        

SimpleCov 1.0 deprecates the older # :nocov: syntax; it still works for now. Read more about skipping code in SimpleCov's readme.

Configuration File

You can configure undercover options for your repository by placing a .undercover file in your repository root. This file uses the same format as the undercover gem's configuration file.

Example .undercover configuration:

          --exclude-files "vendor/*,db/*,bin/*,spec/*"
          --include-files "*.rb,*.rake"
        

UndercoverCI will automatically detect and apply these settings when analyzing your coverage reports. Commonly used options include --exclude-files and --include-files for controlling which files are analyzed. Note that options meant for local undercover usage (like coverage file paths and git settings) will be overridden by UndercoverCI's own configuration. For a complete list of available options, see the undercover gem documentation.

In a monorepo, keep .undercover in the repository root, but write --include-files and --exclude-files globs relative to the directory SimpleCov covered. If tests run in apps/dash, use lib/**/*.rb rather than apps/dash/lib/**/*.rb. This changed in undercover 0.9.0, so drop the app directory from existing globs.

Branch filtering

UndercoverCI allows you to configure which branches should trigger coverage checks using regular expressions. This is useful for excluding certain branches like main, release branches, or including only branches with specific naming patterns.

To configure branch filtering, go to your Settings page and expand the configuration section for any repository. Enter a regular expression in the "Branch Filter" field that matches the branches you want to include for coverage checks.

Examples

Leave the field blank to run coverage checks on all branches. If you enter an invalid regular expression, UndercoverCI will default to checking all branches.

Critical paths

Critical paths let a neutral (🟡) repository fail the check (🔴) when untested changes touch code you care about most, such as billing or authentication. Everything else stays neutral. Warnings in critical paths are annotated as failures in the pull request diff and marked (critical) in the check run table. With the "Failure" mode every warning already fails the check, so critical paths only apply to neutral repositories.

To configure critical paths, go to your Settings page, expand the configuration section for a repository, select "Neutral" and enter one glob per line in the "Critical paths" field. Blank lines and comments starting with # are ignored, on their own line or after a pattern, and up to 50 patterns are allowed.

Patterns are relative to the repository root, not to the directory SimpleCov covered. In a monorepo with tests running in apps/dash, write apps/dash/app/models/billing/**/*. Use **/ to match any number of directories (end a directory pattern with /**/*, a trailing /** only matches one level) and {a,b} to match alternatives.

Examples

Skip coverage entirely

UndercoverCI checks can be skipped entirely if an analysis is redundant for any user-defined reason (e.g. commit is unrelated or tests haven't ran). Use the --cancel uploader option within your CI workflow to achieve this (uploader docs).

API V1

To access Undercover HTTP API, configure your per-user auth token using the API section of the Settings page.

GET /v1/checks/:commit_sha

Returns the coverage check information for the given commit SHA. Example:

          curl -H "Authorization: Bearer $UNDERCOVER_API_TOKEN" \
          https://undercover-ci.com/v1/checks/aa412c105eff7a7af918fc422bf5085101295798
        

Response format:

          {
            "id": 327889,
            "head_sha": "aa412c105eff7a7af918fc422bf5085101295798",
            "base_sha": "master",
            "state": "complete",
            "state_log": [
              {
                "to": "awaiting_coverage",
                "ts": "2023-11-25T15:27:14Z",
                "from": "created"
              },
              {
                "to": "in_progress",
                "ts": "2023-11-25T15:28:48Z",
                "from": "awaiting_coverage"
              },
              {
                "to": "complete",
                "ts": "2023-11-25T15:37:22Z",
                "from": "in_progress"
              }
            ],
            "repo_full_name": "twitchy-tortoise/undercover-test"
          }
        

GET /v1/checks/:commit_sha/coverage

Returns the coverage report for the given commit SHA. The coverage report will be in the original format (JSON or LCOV) as uploaded. The `curl` example below includes the -L flag to ensure curl follows the redirect to the storage provider.

          curl -LH "Authorization: Bearer $UNDERCOVER_API_TOKEN" \
          -o check_coverage.json \
          https://undercover-ci.com/v1/checks/aa412c105eff7a7af918fc422bf5085101295798/coverage
        

---

Development

The UndercoverCI Github App and the underlying undercover gem are developed and available on GitHub under a standard MIT license. Your contributions are welcome!

• grodowski/undercover-ci
• grodowski/undercover

---