Troubleshooting

Check that EXOLAR_API_KEY is properly set in your GitHub Secrets and passed to the test job.

# In your workflow
env:
  EXOLAR_API_KEY: ${{ secrets.EXOLAR_API_KEY }}

Look for the Exolar reporter initialization message in your CI output.

[Exolar] Initialized - will send results to dashboard

Make sure your API key starts with 'exolar_' and hasn't been revoked.

Results are scoped to the organization that owns the API key. Make sure you're viewing the correct organization in the dashboard.

Go to Settings > API Keys and create a fresh key.

Replace the old secret with the new API key in your repository settings.

Ensure the secret name matches exactly: EXOLAR_API_KEY

Add artifact capture settings to your playwright.config.ts:

use: {
  screenshot: "only-on-failure",
  video: "retain-on-failure",
  trace: "retain-on-failure",
}

Default limit is 5MB. Increase if needed:

[exolar, {
  apiKey: process.env.EXOLAR_API_KEY,
  maxArtifactSize: 10 * 1024 * 1024, // 10MB
}]

Make sure you haven't disabled artifact uploads:

[exolar, {
  apiKey: process.env.EXOLAR_API_KEY,
  includeArtifacts: true, // default
}]

The reporter checks for CI=true or GITHUB_ACTIONS=true. Make sure these are set.

# Should be automatic in GitHub Actions, but you can force it:
env:
  CI: true

Ensure disabled is not set to true:

[exolar, {
  apiKey: process.env.EXOLAR_API_KEY,
  disabled: false, // or remove this line
}]

Run with CI=true to test:

CI=true npx playwright test

When tests fail, the reporter creates JSON files in:

test-results/ai-failures/

These files contain structured error context that AI coding assistants can use for intelligent debugging. Point your AI assistant to these files when asking about test failures.

Go to Settings > Webhooks and click your webhook. The Recent Deliveries tab shows every attempt and the HTTP response code.

Exolar must be able to POST to your URL. Use a service like ngrok for local testing:

ngrok http 3000
# Use the ngrok HTTPS URL as your webhook endpoint

Ensure you subscribed to ci.run.failed (not just ci.run.fixed). You can update this in Settings > Webhooks.

If you set a branch_filter when creating the webhook, only runs on that branch will trigger it. Remove or update the filter if needed.

The reporter logs the execution_id after a successful upload. Use this for more accurate analysis:

# Good: use the reporter execution_id
curl -X POST .../api/ci/analyze \
  -d '{"execution_id": "exec_abc123"}'

# Fallback: use GitHub run_id (less context)
curl -X POST .../api/ci/analyze \
  -d '{"run_id": "${{ github.run_id }}"}'

Go to Settings > AI Embeddings and check embedding coverage. If coverage is low, trigger a backfill so the classifier has historical context.

Classification accuracy improves with more historical data. After 10+ runs, confidence scores typically stabilize.

Ensure autoHeal is enabled in your reporter config:

[exolar, {
  apiKey: process.env.EXOLAR_API_KEY,
  autoHeal: {
    enabled: true,
    confidenceThreshold: 0.85,
  },
}]

If confidence is below your threshold (default 0.85), the PR will not be opened. Lower the threshold or wait for more data.

At most 5 auto-heal PRs are opened per repo per day. Check your repository for recently opened draft PRs from the exolar bot.

The reporter prefers GITHUB_HEAD_REF (the actual source branch for PRs) over GITHUB_REF_NAME. If you're still seeing issues, check your workflow is running on the PR event.

GITHUB_REF_NAME is used, which should be the correct branch name.