> ## Documentation Index
> Fetch the complete documentation index at: https://docs.akhara.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Company name is Akhara AI (never Rubric AI). Keep lowercase rubric/rubrics only when meaning grading criteria.
> Expert Review (docs path talent/) is enterprise BYO experts for audit and review: invite customer specialists; do not pitch Akhara recruiting or a public expert career portal. RLHF and domain writing are secondary work types.
> Prefer concrete API examples against public hosts: Environments eval API https://agi.akhara.ai, Control plane PDP https://api.akhara.dev, Evaluation https://app.akhara.ai / https://api.akhara.ai, Expert Review portal https://talent.akhara.ai.
> Do not invent a public hostname for private orchestrators or env API internals.
> Do not confuse control-plane latches with Environments confirmation latches.
> Environments SDK/API examples: curl against https://agi.akhara.ai. Evaluation SDK: from akhara import Akhara and AKHARA_API_KEY.
> Start with /llms.txt for the docs index and OpenAPI links; fetch individual pages as .md exports.

# Evaluations API

> Start scored runs against an environment, poll status, cancel, and open free-play episodes.

Base: `https://agi.akhara.ai`

Pass `env_id` to select which environment runs your tasks (e.g. `amazon_gym_android`, `cvs_gym`).

## List tasks

```http theme={null}
GET /evaluations/tasks
```

```bash theme={null}
curl -s https://agi.akhara.ai/evaluations/tasks | python3 -m json.tool | head
```

Returns the ordered task bank used when you pass `num_examples`.

## Start evaluation

```http theme={null}
POST /evaluations/run
```

```json theme={null}
{
  "env_id": "amazon_gym_android",
  "model": "my-agent-v1",
  "task_ids": ["vu.checkout.t098", "vu.search.add.t309"],
  "rollouts_per_example": 1,
  "webhook_url": null,
  "endpoint": {
    "url": "https://your-agent.example.com",
    "key": null,
    "model": "gpt-4.1",
    "use_wrapper": false,
    "send_tool_definitions": false,
    "use_openai_chat": false,
    "minimal_observation": null,
    "additional_params": null
  }
}
```

| Field                            | Required | Notes                                                    |
| -------------------------------- | -------- | -------------------------------------------------------- |
| `env_id`                         | yes      | Target gym                                               |
| `endpoint.url`                   | yes      | Public HTTPS URL of your agent                           |
| `task_ids` **or** `num_examples` | one of   | Explicit ids or first N bank tasks                       |
| `rollouts_per_example`           | no       | Default `1`, max `200`                                   |
| `model`                          | no       | Label stored on the run                                  |
| `endpoint.use_openai_chat`       | no       | If true, POST OpenAI chat to `{url}/v1/chat/completions` |
| `webhook_url`                    | no       | Optional completion webhook                              |

**Response (202):**

```json theme={null}
{
  "run_id": "e9eb19e7-187d-4bc4-bb59-a187807e4568",
  "status": "RUNNING",
  "status_url": "/evaluations/run/e9eb19e7-187d-4bc4-bb59-a187807e4568/status",
  "message": "Evaluation started. Poll status_url for progress and results."
}
```

## Get run status

```http theme={null}
GET /evaluations/run/{run_id}/status
```

```bash theme={null}
curl -s "https://agi.akhara.ai/evaluations/run/${RUN_ID}/status" | python3 -m json.tool
```

```json theme={null}
{
  "run_id": "e9eb19e7-187d-4bc4-bb59-a187807e4568",
  "status": "COMPLETED",
  "progress": {
    "completed_rollouts": 2,
    "total_rollouts": 2,
    "current_episode_id": null,
    "current_task_id": null
  },
  "env_id": "amazon_gym_android",
  "model": "my-agent-v1",
  "started_at": "2026-07-30T01:00:00Z",
  "completed_at": "2026-07-30T01:12:00Z",
  "evaluation_id": "a1b2c3d4",
  "run_output_path": "outputs/runs/e9eb19e7-187d-4bc4-bb59-a187807e4568",
  "evaluations": [
    {
      "id": "a1b2c3d4",
      "results": [
        {
          "example_id": 0,
          "rollout_id": "e9eb19e7-187d-4bc4-bb59-a187807e4568_0_0",
          "episode_id": "ep_c8c22030d0",
          "task": "vu.checkout.t098",
          "reward": 1.0,
          "task_passed": true,
          "task_outcome": "pass",
          "is_completed": true,
          "is_truncated": false,
          "stop_condition": "terminal_action",
          "verifiers": [],
          "error": null
        }
      ]
    }
  ]
}
```

### Result fields that matter

| Field            | Use                                |
| ---------------- | ---------------------------------- |
| `reward`         | Terminal score `+1` / `0` / `-1`   |
| `task_passed`    | Boolean pass                       |
| `rollout_id`     | Key for dashboard trajectory fetch |
| `episode_id`     | Orchestrator / export id           |
| `verifiers`      | Final verifier objects             |
| `stop_condition` | Why the episode ended              |

## Cancel run

```http theme={null}
DELETE /evaluations/run/{run_id}
```

## List / get evaluations

```http theme={null}
GET /evaluations?environment_id=amazon_gym_android&limit=20
GET /evaluations/{evaluation_id}
```

## Open run

Unscored free-play with a custom instruction (no verifiers, reward `0.0`).

```http theme={null}
POST /runs/open
```

```json theme={null}
{
  "env_id": "amazon_gym_android",
  "task_definition": "Browse headphones and add any pair to cart.",
  "max_steps": 20,
  "profile_id": "U_001",
  "rollouts": 1,
  "endpoint": {
    "url": "https://your-agent.example.com"
  }
}
```

Poll with the same `GET /evaluations/run/{run_id}/status`.

## Agent observation (what we POST to you)

Typical fields your endpoint receives (wrapper `AgentObservation`):

* `screenshot_b64`
* `instruction`, `task_id`, `step`, `episode_id`
* UI / screen metadata (and optionally slim mode via `minimal_observation`)

Return a JSON action (`action` + `params`) as in [Actions](/environments/mechanics/actions).