> ## Documentation Index
> Fetch the complete documentation index at: https://docs.akhara.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Company name is Akhara AI (never Rubric AI). Keep lowercase rubric/rubrics only when meaning grading criteria.
> Expert Review (docs path talent/) is enterprise BYO experts for audit and review: invite customer specialists; do not pitch Akhara recruiting or a public expert career portal. RLHF and domain writing are secondary work types.
> Prefer concrete API examples against public hosts: Environments eval API https://agi.akhara.ai, Control plane PDP https://api.akhara.dev, Evaluation https://app.akhara.ai / https://api.akhara.ai, Expert Review portal https://talent.akhara.ai.
> Do not invent a public hostname for private orchestrators or env API internals.
> Do not confuse control-plane latches with Environments confirmation latches.
> Environments SDK/API examples: curl against https://agi.akhara.ai. Evaluation SDK: from akhara import Akhara and AKHARA_API_KEY.
> Start with /llms.txt for the docs index and OpenAPI links; fetch individual pages as .md exports.

# Scores and stats

> How to retrieve rewards, pass rates, trajectories, and aggregate metrics.

## Live evaluation scores

After `POST /evaluations/run`, poll:

```http theme={null}
GET /evaluations/run/{run_id}/status
```

Per-rollout fields:

| Field            | Meaning                                                      |
| ---------------- | ------------------------------------------------------------ |
| `reward`         | Terminal hybrid score: `+1` pass, `0` incomplete, `-1` fail  |
| `task_passed`    | Boolean derived from verifiers / stop                        |
| `task_outcome`   | Human-readable outcome (`pass`, `fail`, …)                   |
| `verifiers`      | Final VU / Room objects                                      |
| `is_truncated`   | Hit `max_steps` without clean terminal                       |
| `stop_condition` | e.g. `terminal_action`                                       |
| `metrics`        | Extra counters (do not confuse shaping with terminal reward) |

List historical evals:

```http theme={null}
GET /evaluations?environment_id=amazon_gym_android&limit=20
```

## Trajectory data

| Need                    | Endpoint                                                                  |
| ----------------------- | ------------------------------------------------------------------------- |
| Full step list          | `GET /dashboard/runs/{run_id}/rollouts/{rollout_id}`                      |
| Step screenshot         | `GET /dashboard/runs/{run_id}/rollouts/{rollout_id}/steps/{i}/screenshot` |
| Run + all rollouts      | `GET /dashboard/runs/{run_id}`                                            |
| Episode zip (post-sync) | `GET /v1/export/run/{episode_id}` on private orchestrator                 |

`rollout_id` = `{run_id}_{example_idx}_{rollout_idx}`.

On-disk capture (operator side) also lands under `outputs/runs/{run_id}/` when the wrapper is configured to write run capture.

## Aggregate stats

```bash theme={null}
curl -s https://agi.akhara.ai/dashboard/stats
```

```json theme={null}
{
  "total_runs": 42,
  "active_runs": 1,
  "total_rollouts": 500,
  "completed_rollouts": 480,
  "failed_rollouts": 12,
  "success_rate": 0.3125,
  "avg_reward": 0.05
}
```

### Pass\@N

```http theme={null}
GET /dashboard/runs/{run_id}/pass_at_n
GET /dashboard/tasks/pass_at_n_calibration
```

Use these when `rollouts_per_example > 1`.

## Offline sample pack metrics

The shopping-environment sample pack ships precomputed metrics:

| File                             | Contents                                       |
| -------------------------------- | ---------------------------------------------- |
| `metrics/summary.json`           | `pass_rate`, `outcome_counts`, `family_counts` |
| `metrics/task_results.csv`       | Per-episode table                              |
| `episode_index.jsonl`            | One row per episode                            |
| `trajectories/.../metadata.json` | `terminal_reward`, `final_verifier_verdicts`   |

Example `summary.json`:

```json theme={null}
{
  "episodes": 12,
  "tasks": 12,
  "pass_rate": 0.25,
  "failure_rate": 0.75,
  "outcome_counts": { "FAIL": 9, "PASS": 3 },
  "family_counts": {
    "checkout": 5,
    "reasoning_checkout": 2,
    "search_add_checkout": 5
  }
}
```

## Interpreting a PASS

A checkout PASS requires more than reaching `ORDER_CONFIRMATION`:

1. Primary VU (e.g. `VU-CHECKOUT`) status `PASS`
2. `ROOM_DEVICE_DB` `PASS` when hybrid Room scoring is on
3. Confirmation / preselection gates satisfied

See [Verifiers](/environments/mechanics/verifiers) and the worked example in [Sample trajectories](/environments/results/sample-trajectories).