Quickstart

This guide uses the managed evaluation API, the hosted path served at https://agi.akhara.ai.

Surface	Production URL
Eval API	`https://agi.akhara.ai`
Admin dashboard	`https://admin-agi.akhara.ai`
Agent endpoint you expose	Public HTTPS URL the platform can reach (e.g. `https://your-agent.example.com`)

Pick an environment with env_id (e.g. amazon_gym_android or cvs_gym).

1. Confirm the platform is up

curl -s https://agi.akhara.ai/health
# {"status":"ok","service":"amazon_gym_rubric_wrapper"}

List tasks for the default bank:

curl -s https://agi.akhara.ai/evaluations/tasks | python3 -m json.tool | head

2. Expose an agent endpoint

Your agent receives an observation (screenshot, instruction, UI metadata) and returns an action. The platform supports:

Raw observation JSON → POST to endpoint.url
OpenAI chat → set use_openai_chat: true (POSTs to {url}/v1/chat/completions)

The platform must be able to reach your agent over the public internet (HTTPS recommended). Minimal action shapes (Android real-app mode):

{ "action": "click", "params": { "point_2d": [540, 960] } }

{ "action": "type", "params": { "text": "hoodie" } }

{ "action": "finished", "params": { "text": "Ordered red hoodie." } }

See Actions for the full surface.

3. Start an evaluation

curl -s -X POST https://agi.akhara.ai/evaluations/run \
  -H 'Content-Type: application/json' \
  -d '{
    "env_id": "amazon_gym_android",
    "model": "my-agent-v1",
    "task_ids": ["vu.checkout.t098"],
    "rollouts_per_example": 1,
    "endpoint": {
      "url": "https://your-agent.example.com",
      "model": "my-agent-v1"
    }
  }' | python3 -m json.tool

Response:

{
  "run_id": "e9eb19e7-187d-4bc4-bb59-a187807e4568",
  "status": "RUNNING",
  "status_url": "/evaluations/run/e9eb19e7-187d-4bc4-bb59-a187807e4568/status",
  "message": "Evaluation started. Poll status_url for progress and results."
}

Provide either task_ids or num_examples (first N tasks from that environment’s bank).

4. Poll status and scores

RUN_ID=e9eb19e7-187d-4bc4-bb59-a187807e4568
curl -s "https://agi.akhara.ai/evaluations/run/${RUN_ID}/status" | python3 -m json.tool

When status is COMPLETED, each result row includes:

Field	Meaning
`reward`	Terminal score: `+1` / `0` / `-1`
`task_passed`	Boolean pass
`task_outcome`	e.g. `pass` / `fail`
`verifiers`	Final verifier verdicts
`rollout_id`	Dashboard trajectory key
`episode_id`	Per-attempt episode id

5. Fetch the trajectory

RUN_ID=...
ROLLOUT_ID="${RUN_ID}_0_0"

curl -s "https://agi.akhara.ai/dashboard/runs/${RUN_ID}/rollouts/${ROLLOUT_ID}" \
  | python3 -m json.tool | head -80

Browse the same run in the UI: https://admin-agi.akhara.ai/runs/{run_id}. For aggregate stats:

curl -s https://agi.akhara.ai/dashboard/stats | python3 -m json.tool

Details: Scores and stats.

Direct environment loop

The low-level env API (/v1/env/reset, /v1/env/step) is not exposed on the public internet. Production clients should use the managed evaluation API above. Operators with private network access can drive episodes directly. See Env API.

Confirmation latches: needs_confirmation before place order / remove / cancel
Architecture: wrapper, orchestrator, workers, devices
Task definitions: how tasks are specified
Sample trajectories: offline ATIF examples

Getting started

Mechanics

API reference

Scores and trajectories

1. Confirm the platform is up

2. Expose an agent endpoint

3. Start an evaluation

4. Poll status and scores

5. Fetch the trajectory

Direct environment loop

Next

​1. Confirm the platform is up

​2. Expose an agent endpoint

​3. Start an evaluation

​4. Poll status and scores

​5. Fetch the trajectory

​Direct environment loop

​Next

1. Confirm the platform is up

2. Expose an agent endpoint

3. Start an evaluation

4. Poll status and scores

5. Fetch the trajectory

Direct environment loop

Next