archal trace-source

Usage

archal trace-source connect <provider> [options]
archal trace-source import <path> [options]
archal trace-source list [options]
archal trace-source test [source] [options]
archal trace-source sync [source] [options]
archal trace-source serve [source] [options]
archal trace-source use <source> [options]
archal trace-source disable <source> [options]

trace-source is the onboarding layer for traces that already exist outside Archal. It creates a small repo-local source registry and writes complete archal.trace.upload-batch JSON files into a trace directory. You can attach one source or several, switch the default source, and keep each source pointed at the same grading directory or separate directories. This command is the supported bridge for traces that are not already in Supabase/Postgres. Supabase/Postgres can be registered for hosted polling with archal attach --source ...; file, HTTP/OTel, Langfuse, Braintrust, S3/GCS, and custom traces are normalized locally and can be uploaded into hosted Attach with --upload --repository owner/repo. Hosted upload requires an authenticated workspace. Run archal login locally, or set ARCHAL_TOKEN to a workspace API key in CI.

Providers

Provider	Best for
`file`	One-time uploads, exported JSON, JSONL, or NDJSON
`http`	Continuous trace pushes from a custom exporter
`otel`	OpenTelemetry or OTLP JSON trace exporters
`langfuse`	Langfuse trace exports or API reads
`braintrust`	Braintrust rows, experiments, or log exports
`supabase`	Supabase-hosted `ai_traces` / `ai_spans` tables
`postgres`	Read-only Postgres tables or SQL views
`s3`	Object-store exports, manifests, or local S3 mirrors
`gcs`	GCS exports, manifests, or local GCS mirrors
`custom`	Homegrown observability pipelines and custom JSON shapes

Quick paths

Upload a file

archal trace-source import ./exports/agent-trace.json --out .archal/traces/inbox
archal trace-source import ./exports --format jsonl --out .archal/traces/inbox
archal login
archal trace-source import ./exports --upload --repository acme/harness

Use --preview before writing files:

archal trace-source import ./exports --preview --json

Continuous HTTP push

archal trace-source connect custom --name "prod exporter" --out .archal/traces/inbox
archal trace-source serve "prod exporter" --port 4319

Then point your exporter at:

http://127.0.0.1:4319/

For OTLP-style payloads:

archal trace-source connect otel --name "otel collector" --out .archal/traces/inbox
archal trace-source serve "otel collector" --port 4319 --path /v1/traces

If you bind beyond localhost, require a receiver token:

export TRACE_RECEIVER_TOKEN=...
archal trace-source serve "prod exporter" \
  --host 0.0.0.0 \
  --api-key-env TRACE_RECEIVER_TOKEN

For a hosted-style receiver, require workspace, source, and idempotency metadata on every push:

export TRACE_RECEIVER_TOKEN=...
archal trace-source serve "prod exporter" \
  --host 0.0.0.0 \
  --api-key-env TRACE_RECEIVER_TOKEN \
  --workspace-id ws_trellis_demo \
  --source-id http-prod-exporter \
  --require-hosted-contract

Push one trace:

curl -X POST http://127.0.0.1:4319/v1/traces \
  -H "Authorization: Bearer $TRACE_RECEIVER_TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Archal-Workspace-Id: ws_trellis_demo" \
  -H "X-Archal-Trace-Source-Id: http-prod-exporter" \
  -H "X-Archal-Trace-Group: production" \
  -H "Idempotency-Key: trace-$(date +%s)" \
  --data @./trace.json

Or wrap the payload explicitly:

{
  "contract": { "schema": "archal.trace.http-push", "version": 1 },
  "workspaceId": "ws_trellis_demo",
  "sourceId": "http-prod-exporter",
  "traceGroup": "production",
  "format": "raw",
  "trace": {
    "traceId": "trace_123",
    "entries": []
  }
}

Retries are safe when they reuse the same Idempotency-Key and identical body. If the same key is reused with different content, Archal rejects the request instead of silently overwriting the trace.

Langfuse or Braintrust

export LANGFUSE_READ_KEY=...
archal trace-source connect langfuse \
  --base-url https://cloud.langfuse.com \
  --api-key-env LANGFUSE_READ_KEY \
  --out .archal/traces/inbox

archal trace-source test langfuse
archal trace-source sync langfuse
archal login
archal trace-source sync langfuse --upload --repository acme/harness

export BRAINTRUST_API_KEY=...
archal trace-source connect braintrust \
  --base-url https://api.braintrust.dev \
  --api-key-env BRAINTRUST_API_KEY \
  --project support-agent \
  --out .archal/traces/inbox

Supabase or Postgres

Store a read-only connection string in an environment variable. The registry stores the variable name, not the secret value.

export TRACE_READONLY_DATABASE_URL=postgres://readonly:...@db.example.com/app
archal trace-source connect postgres \
  --name "agent traces db" \
  --database-url-env TRACE_READONLY_DATABASE_URL \
  --traces-table ai_traces \
  --spans-table ai_spans \
  --cursor-column created_at \
  --out .archal/traces/inbox

archal trace-source test "agent traces db"
archal trace-source sync "agent traces db" --limit 50

For Trellis-style Supabase/Postgres traces, keep the default table names ai_traces and ai_spans. Archal maps top-level run fields such as workspace_id, trace_group, status, session_id, agent_id, conversation_id, model_id, token/cost counts, input/output data, metadata, errors, tags, and classifications. Child span rows preserve trace_id, parent_span_id, span_type, timing, token/cost fields, previews, full JSON input/output, and errors. Preview before writing files:

archal trace-source sync "agent traces db" --limit 50 --preview --json

Custom read-only views work too:

archal trace-source connect postgres \
  --name "custom trace view" \
  --database-url-env TRACE_READONLY_DATABASE_URL \
  --query "select * from agent_trace_export order by created_at desc" \
  --out .archal/traces/inbox

S3 or GCS exports

Use a local mirror for fastest setup. --bucket and --prefix document the source, while --path tells Archal where the exported objects are available on this machine.

archal trace-source connect s3 \
  --bucket acme-agent-traces \
  --prefix prod/ \
  --path ./trace-export-mirror \
  --out .archal/traces/inbox

archal trace-source sync "s3 traces" --limit 100 --upload --repository acme/harness

For manifest-driven exports, pass a JSON manifest URL. If the manifest URL is signed, keep it in an environment variable and pass --manifest-url-env.

archal trace-source connect gcs \
  --bucket acme-agent-traces \
  --manifest-url https://storage.googleapis.com/acme-agent-traces/manifest.json \
  --out .archal/traces/inbox

export GCS_TRACE_MANIFEST_URL=...
archal trace-source connect gcs \
  --bucket acme-agent-traces \
  --manifest-url-env GCS_TRACE_MANIFEST_URL \
  --out .archal/traces/inbox

Source switching

archal trace-source list
archal trace-source use "agent traces db"
archal trace-source disable "old exporter"

This lets one team start with file uploads, move to an HTTP exporter, then move again to a database or object-store source without changing the downstream grading directory.

Safety

Credentials are stored as environment variable names, not raw secret values.
Signed URLs and raw API keys are rejected from the source registry; store them in environment variables instead.
Redaction runs before preview, file write, and provider metadata persistence.
File writes use a temporary file and atomic rename, so readers never see a partial trace file.
Postgres sources require a read-only SELECT; table names and cursor columns are quoted and validated before querying.
Each normalized file includes a stable content hash and idempotency key.

Documentation Index

​Usage

​Providers

​Quick paths

​Upload a file

​Continuous HTTP push

​Langfuse or Braintrust

​Supabase or Postgres

​S3 or GCS exports

​Source switching

​Safety