CI integration

Archal runs in any CI environment. Set the auth token, pick an output format, set a pass threshold. The build fails if satisfaction drops below it. Run npx archal init locally first, then commit .archal.json, your harness, and the archal devDependency. CI should use the project-pinned CLI with npx archal.

Secrets

The only required secret is your Archal token. For CI and any other non-interactive environment, use a workspace API key (archal_ws_...) set as ARCHAL_TOKEN:

ARCHAL_TOKEN=archal_ws_<your-key>

If your harness calls a model provider directly, pass the provider key it reads:

OPENAI_API_KEY=sk-...
# or ANTHROPIC_API_KEY, GEMINI_API_KEY, etc.

Creating an API key

Create a workspace API key with archal workspace api-key create ci-runner from a logged-in owner/admin session, or from the dashboard: Settings > API Keys. Copy it once and store it as a CI secret. List or revoke keys with archal workspace api-keys and archal workspace api-key revoke <key-id>. Browser login (archal login) also authenticates the CLI, but its session token is short lived, so prefer an API key for CI. See archal workspace. A workspace API key is already bound to one workspace. If you use a user token and belong to more than one workspace, set ARCHAL_WORKSPACE_ID to target a specific one:

ARCHAL_WORKSPACE_ID=<workspace-id>

GitHub Actions

name: Agent tests
on: [push]

jobs:
  archal:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 22

      - name: Install dependencies
        run: npm ci

      - name: Run scenarios
        env:
          ARCHAL_TOKEN: ${{ secrets.ARCHAL_TOKEN }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          npx archal run scenarios/close-stale-issues.md \
            --docker \
            --runs 3 \
            --pass-threshold 80 \
            -o json \
            -q

GitLab CI

archal:
  image: node:22
  services:
    - docker:dind
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
    ARCHAL_TOKEN: $ARCHAL_TOKEN
    OPENAI_API_KEY: $OPENAI_API_KEY
  script:
    - npm ci
    - >
      npx archal run scenarios/close-stale-issues.md
      --docker
      --runs 3
      --pass-threshold 80
      -o json
      -q

Useful flags

Flag	What it does
`--pass-threshold <score>`	Exit 1 if satisfaction is below this (0-100)
`-o json`	Machine-readable JSON output
`-q`	Suppress non-error output
`-n, --runs <count>`	Run the scenario multiple times for a real satisfaction score
`--tag <tag>`	Only run scenarios with a matching tag (exits 0 if no match)

Exit codes

Code	Meaning
`0`	Score met the threshold (or scenario skipped by `--tag`)
`1`	Score below threshold or runtime error
`2`	Validation error (bad flags, missing scenario, invalid config)

Go deeper

Writing scenarios covers repeatable checks.
Security covers tokens, telemetry, and trace upload.
CLI output covers JSON output and CI-friendly flags.

Run multiple scenarios

List scenario paths in the scenarios array of .archal.json. archal run executes each one:

{
  "agent": {
    "command": "npx",
    "args": ["tsx", "src/agent.ts"]
  },
  "clones": ["github"],
  "scenarios": [
    "scenarios/close-stale-issues.md",
    "scenarios/triage-new-issues.md"
  ],
  "runs": 3,
  "timeout": 120
}

Start here

Test agents

Run anywhere

Advanced

CI integration

Secrets

Creating an API key

GitHub Actions

GitLab CI

Useful flags

Exit codes

Go deeper

Run multiple scenarios

​Secrets

​Creating an API key

​GitHub Actions

​GitLab CI

​Useful flags

​Exit codes

​Go deeper

​Run multiple scenarios

Secrets

Creating an API key

GitHub Actions

GitLab CI

Useful flags

Exit codes

Go deeper

Run multiple scenarios