# Test runs

> Source: https://astrallab.dev/docs/test-runs

A test run turns a script into evidence. The agent wraps a flow in `astral.test`, splits it into steps, states what should be true, and Astral records what happened.

Test runs use the `run_script` tool, which runs agent-written JavaScript in a sandbox that calls the other tools. The `script` group is off by default; the `tests` group is on by default but only takes effect while `script` is on. See [the script group](https://astrallab.dev/docs/tools.md#group-script).

## An example

```js
return await astral.test('Login demo', async () => {
  await astral.step('Open the store', () => astral.navigate({url: 'http://localhost:3000/'}));
  await astral.step('Sign in', async () => {
    await astral.type({locator: {role: 'textbox', name: 'Username'}, text: 'demo'});
    await astral.type({locator: {role: 'textbox', name: 'Password'}, text: 'demo', submit: true});
    await astral.expect.visible({role: 'button', name: 'Sign out'});
    await astral.expect.noErrors();
  });
  await astral.step('Pay', async () => {
    await astral.click({locator: {role: 'button', name: 'Pay'}});
    await astral.expect.request({urlContains: '/api/', method: 'POST', status: '2xx'});
  });
}, {continueOnFailure: true, record: 'actions'});
```

Give `run_script` a `timeoutMs` of 60000 to 120000 for a test: evidence takes time.

## astral.test and astral.step

`astral.test(name, fn, options)` options: `tabId`, `continueOnFailure` (default false) and `record`. It does not throw when a step fails: check the `status` in the result, which also has the totals, a Markdown `summary`, and the paths of the report.

`astral.step(name, fn)` groups actions. At its end, and always on failure, Astral saves a screenshot, the URL and title, the requests made during the step (failing ones with headers and bodies redacted), console errors and warnings, and any new error groups. A step fails when its function throws or an expectation in it fails. Without `continueOnFailure` the test stops at the first failed step.

`run_script` also accepts `test: {name, continueOnFailure, ...}` to treat the whole script as one test.

## Expectations

| Call | Passes when |
| --- | --- |
| `expect.visible(locator)` | The element is visible, waiting up to `timeoutMs` (default 5000). |
| `expect.hidden(locator)` | The element is not visible. |
| `expect.text(locator, 'contains')` / `expect.text('text')` | The element contains the text, or the text appears anywhere on the page. |
| `expect.url('/dashboard')` / `expect.url(/regex/)` | The tab's URL matches. |
| `expect.request({urlContains, method, status})` | A matching request was made since the step started. `status` is a number, `'2xx'` or `'error'`. |
| `expect.noErrors({kinds, waitMs, ignoreStatus, ignore})` | No new error group appeared since the step started. Ignore statuses like 401 or messages by substring. |
| `expect.layout({rules, ignore})` | The page's responsive layout has no errors at the run's viewport. See [the layout check](https://astrallab.dev/docs/saved-tests.md#layout). |

A locator is `{role, name, text, nth, exact}`, a snapshot ref such as `"e12"`, or a text. A failed expectation throws an assertion error with what was expected and what was found, and the report shows it. `astral.screenshot({name, fullPage})` inside a test adds a named screenshot to the current step.

## Record modes

| Mode | What is captured |
| --- | --- |
| `'off'` | Screenshots only of failures. |
| `'steps'` (default) | One screenshot at the end of every step. |
| `'actions'` | Also one after every click, type, select, key press and navigation, taken while a marker shows the target: a box, a ripple at the click point and a caption such as _click "Sign in"_. |
| `'video'` | Also frames of the tab at 4 per second (up to 5 minutes or 200 MB), saved as an animated `recording.png`, and `recording.mp4` where Astral can encode one (macOS), with a frame player in the report. |

Markers are drawn in every mode except `'off'`, disappear after 2.5 seconds and at the end of the run, and never catch clicks. Text typed into a password, one-time-code or card field is never written: captions show •••••• and the value is masked everywhere else in the report. See [Platforms](https://astrallab.dev/docs/platforms.md) for video support.

## What a run leaves

Each run writes a folder in Astral's data directory containing:

- `report.html`: a readable report with a summary, a card per step with its screenshots, the failed assertion, failing requests, console errors, grouped page errors, an actions timeline and a performance section.
- `report.json`: the same data, plus a Markdown summary written to be pasted into an issue.
- The screenshots as PNG files, and the recording if you asked for one.

The newest 50 runs are kept (configurable in Settings → Developer, where test runs can also be turned off). The agent reads runs back with `test_runs_list` and `test_run_get`.

## Evidence for tickets

The Markdown summary is meant to be pasted into an issue, next to the screenshots and the report. If your agent is connected to your ticket tool, it can post that evidence there for you. Astral itself does not integrate with any ticket tool: it writes the evidence and leaves the posting to you or your agent.

Test names, steps, screenshot names and everything Astral writes about a run are in English, whatever language you, the ticket or the page use. Only locators keep the page's own text.