Running in CI

The point of running Atlas from CI is to catch a performance regression the same way you catch a failing unit test — automatically, on the build that caused it.

A complete example

export ATLAS_URL=https://atlas.cloudbeat.io
export ATLAS_TOKEN=atlas_pat_...

# Only if the plan or its data live in this repository and may have changed.
atlas files upload -p SHOP ./checkout.jmx ./data/users.csv

atlas run -p SHOP --jmx checkout.jmx --servers 5 \
          --gate "error_rate<1,p95<2000" \
          --junit results.xml

If your test plan is maintained in Atlas rather than in the repository, drop the upload step — the run command works on its own.

Pass/fail thresholds

--gate fails the command when a threshold is breached, checked against the run's overall totals.

Metric	Meaning
error_rate	percentage of failed requests
avg	average response time, ms
p90, p95, p99	percentile response times, ms
max	slowest response, ms
throughput	requests per second
requests	total requests

Use <, <=, > or >=. Separate several with commas, or repeat --gate.

--gate "error_rate<1,p95<2000,throughput>50"

Setting a gate implies --wait — there is nothing to check until the test has finished.

Which thresholds to start with

error_rate and a percentile. Averages hide the tail: a p95 catches the slow requests a handful of users actually felt, which an average smooths away.

Exit codes

Code	Meaning
0	the test ran and every threshold passed
1	the test ran but a --gate threshold was breached
2	something was wrong with the request — bad option, unknown project, not signed in
3	the test failed or timed out, or the server could not be reached

The distinction that matters: 1 means your system was too slow, 2 and 3 mean the test never produced a verdict. Treating them the same makes a broken pipeline look like a performance regression.

JUnit report

atlas run -p SHOP --servers 5 --junit results.xml

Writes a standard JUnit file with one test case per threshold, so CI shows each one as a separate pass or fail. The run's headline numbers are included as properties. Point your CI's test reporter at it and gates appear alongside your other tests.

Example: GitHub Actions

- name: Load test
  env:
    ATLAS_URL: https://atlas.cloudbeat.io
    ATLAS_TOKEN: ${{ secrets.ATLAS_TOKEN }}
  run: |
    npm install -g @cloudbeat/atlas-cli
    atlas run -p SHOP --servers 5 \
              --gate "error_rate<1,p95<2000" \
              --junit results.xml

- name: Publish results
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: load-test
    path: results.xml

Things worth knowing

Load tests take minutes, not seconds. Give the job a generous timeout, and consider running it on a schedule or on merges to main rather than on every push.

Machines may be busy. If another test holds the generators you asked for, your run is queued and starts when they free up. Set --timeout to bound how long you're willing to wait.

A queued or failed run is not a regression. That's the exit-code distinction above — don't let a 3 open a performance bug.