Publishing JUnit XML test results in CircleCI transforms raw test output into actionable insights, allowing you to quickly identify failing tests and track historical trends.
Here’s how it works with a sample workflow:
version: 2.1
jobs:
test:
docker:
- image: cimg/go:1.17
steps:
- checkout
- run:
name: Run Tests and Generate JUnit XML
command: |
go test -v ./... -json | go-junit-report > report.xml
- store_test_results:
path: report.xml
workflows:
build-and-test:
jobs:
- test
When this workflow runs, the go test command executes your Go tests. The -json flag outputs results in a machine-readable JSON format. go-junit-report then transforms this into the standard JUnit XML format, which CircleCI understands. The store_test_results step uploads this report.xml file.
In the CircleCI UI, after the job completes, you’ll see a "Test Results" tab on the build details page. This tab presents a summary of your test runs: how many passed, failed, and errored. Clicking into it reveals individual test cases, their status, and any error messages or stack traces for failures. This granular view is crucial for debugging.
The core problem this solves is visibility. Without publishing test results, a CI pipeline might pass even if tests are failing silently, or you’d have to manually sift through logs for each build. By standardizing on JUnit XML, CircleCI can parse, aggregate, and display this information consistently.
The store_test_results command is the key integration point. It tells CircleCI which files contain the test reports. CircleCI then parses these files, extracts the relevant data, and makes it available through its API and UI. This allows for features like:
- Test Summary: A quick overview of test execution status for each build.
- Flaky Test Detection: Over time, CircleCI can help identify tests that pass and fail intermittently.
- Performance Tracking: While not directly part of JUnit XML, the structured data can be a basis for tracking test execution times.
- Integration with other tools: The parsed results can be used by other services or for custom reporting.
The path parameter in store_test_results can accept a single file or a glob pattern (e.g., test-reports/*.xml) to upload multiple report files from a directory. CircleCI will process all matching files.
The go-junit-report tool is just one example. Many testing frameworks and languages have similar tools or built-in options to generate JUnit XML. For example, in Java with Maven, you’d typically find reports in target/surefire-reports/TEST-*.xml. You would then configure store_test_results to point to that directory.
The most surprising aspect for many is how store_test_results is idempotent and efficient. You can run it multiple times within a single job, or even if the files haven’t changed since the last upload, and CircleCI handles it gracefully. It doesn’t re-upload unchanged data, and it correctly merges results if multiple store_test_results commands are used.
Beyond just seeing failures, you can leverage the historical data. CircleCI’s UI allows you to see test results across different builds, helping you spot regressions or improvements in test stability and performance over time.
The next step is to integrate this with your branch protection rules, ensuring that branches cannot be merged if there are failing tests.