# Run in CI

> Source: https://astrallab.dev/docs/ci

`astral-test` runs a project's saved tests without opening Astral and without bundling or downloading a browser. Use it in CI, or in an agent's loop to re-run tests after a fix.

## Install

`astral-test` is a single binary for macOS (Apple Silicon and Intel) and Linux x64, linked from the [developer downloads](https://astrallab.dev/#developer-downloads). A Windows build may be missing from a release because Windows is [not verified yet](https://astrallab.dev/docs/platforms.md).

```bash
BASE=https://downloads.perseusoft.tech/astral-test/latest
curl -fsSLO "$BASE/astral-test-x86_64-unknown-linux-gnu.tar.gz"
tar xzf astral-test-x86_64-unknown-linux-gnu.tar.gz
./astral-test --version
```

It needs a browser already on the machine: Google Chrome, Chromium or Microsoft Edge (the `cdp` engine; `ASTRAL_CHROME` points at a specific binary), any browser already started with DevTools through `--cdp-url`, or Lightpanda in your `PATH` (the `lite` engine). `astral-test doctor` reports what it found and what it would use.

## Commands

```
astral-test run [PATH=.] [--env NAME] [--test ID]... [--collection ID]
                [--engine lite|cdp] [--cdp-url ws://...|http://...]
                [--reporter pretty|json|junit]... [--out DIR=astral-test-results]
                [--timeout-ms N=120000] [--bail] [--record off|steps|actions|video]
astral-test list [PATH=.] [--json]
astral-test doctor [PATH=.] [--engine lite|cdp] [--cdp-url ...] [--probe] [--json]
```

- With no `--test` or `--collection`, every saved test runs, ordered by id.
- `--env` takes an environment of `config.json` or an http(s) URL.
- `--bail` reports the rest as skipped after the first failure.
- Each test runs in a fresh browser context: no cookies or storage carry over.

### Exit codes

| Code | Meaning |
| --- | --- |
| 0   | Every selected test passed (skipped ones do not fail). |
| 1   | At least one test failed: the app did not do what the test expects. |
| 2   | Usage or configuration: bad arguments, project, `config.json`, unknown test or environment, a credential that cannot be resolved. |
| 3   | Engine: no browser found, or it would not start, answer or stay alive. |

With several problems, the most serious wins: 3, then 2, then 1.

## Engines

|     | `lite` (Lightpanda) | `cdp` (Chrome, Chromium, Edge) |
| --- | --- | --- |
| Fidelity | DOM and JavaScript, no rendering or CSS layout. | Full. |
| Evidence | Page trees instead of screenshots; no video. | Screenshots and an animated-PNG recording. |
| Layout checks | Reported as skipped, never passed. | Yes, with viewport emulation. |
| Use | Fast flows with no visual dependency. | Layout and screenshot-dependent tests. |

By default `lite` is used if Lightpanda is installed, otherwise `cdp`.

## Credentials in CI

`{ "env": "ASTRAL_TEST_..." }` reads a process environment variable, which in CI is a secret. `{ "local": "..." }` reads `.astral/secrets.local.json`. A `keychain` reference is not available outside the app: the test fails with a message saying which variable to set. Secret values never appear in the output.

In CI the repository is the source of truth: on a protected environment, `astral-test` trusts the `readOnly` declaration in the test file, so a change under `.astral/` needs code review.

## Output

### JUnit and artifacts

With `--reporter junit` and `--out DIR` you get:

- `junit.xml`, one test case per test (and per viewport, suffixed like `[mobile]`).
- `<testId>/report.html` and `report.json`, the same report the app writes.
- Screenshots (or page trees on `lite`) and the full page tree at the point of a failure.

### NDJSON for agents

`--reporter json` prints one JSON object per line on stdout, flushed as it goes. Every line has `type`, `schemaVersion` (currently 1) and `ts`.

```
run_start
  test_start                (per test)
    step_start / step_end   (per step, live)
  test_end                  (status, steps, failure, artifacts)
run_end                     (always the last line)
```

A failed test's `failure` says what step failed and why: its kind (assertion, action, script, timeout, config, engine or stopped), the expectation, what was expected and what was found, the target element, the URL, a short excerpt of the page tree around the element, console errors and failing requests, all with secrets masked. `failed` is a verdict on your app; `error` means the test could not run as configured.

## GitHub Actions

```yaml
name: e2e
on: [push, pull_request]
jobs:
  astral-test:
    runs-on: ubuntu-latest          # Google Chrome is preinstalled
    steps:
      - uses: actions/checkout@v4
      - name: Install astral-test
        run: |
          curl -fsSL https://downloads.perseusoft.tech/astral-test/latest/astral-test-x86_64-unknown-linux-gnu.tar.gz \
            | sudo tar xz -C /usr/local/bin astral-test
      - name: Start the app under test
        run: npm ci && npm start &
      - name: Run saved tests
        run: astral-test run . --collection smoke --engine cdp --reporter pretty --reporter junit --out astral-test-results
        env:
          ASTRAL_TEST_SHOP_DEMO_PASSWORD: ${{ secrets.SHOP_DEMO_PASSWORD }}
      - if: always()
        uses: actions/upload-artifact@v4
        with: { name: astral-test-results, path: astral-test-results }
```

On GitLab, publish `astral-test-results/junit.xml` under `artifacts: reports: junit`.