Sample trajectories

Example trajectories from the mobile shopping environment ship at outputs/akhara_harbor_samples_v3/ (zip: outputs/akhara_harbor_samples_v3.zip). Trajectories are ATIF-v1.7 with screenshots and UI states on disk. Other environments use the same export shape.

Pack layout

akhara_harbor_samples_v3/   # directory name is historical
  README.md
  episode_index.jsonl
  task_index.jsonl
  metrics/summary.json
  metrics/task_results.csv
  tasks/<task_id>/
    task_definition.json
    verifier_spec.json
  trajectories/<task_id>__<episode_id>/
    trajectory.json          # ATIF-v1.7
    metadata.json
    verifier_results.jsonl
    screenshots/
    ui_states/
    terminal_state.json
  environment/

Slim excerpts used in this docs site live under /samples/.

Highlight: successful checkout (T098)

Field	Value
Task	`vu.checkout.t098`: “Find me a red hoodie and checkout.”
Episode	`ep_c8c22030d0`
Steps	12
Outcome	PASS (`terminal_reward: 1.0`)
Terminal screen	`ORDER_CONFIRMATION`
Verifiers	`VU-CHECKOUT` + `ROOM_DEVICE_DB` PASS

Metadata

{
  "episode_id": "ep_c8c22030d0",
  "task_id": "vu.checkout.t098",
  "outcome": "PASS",
  "terminal_reward": 1.0,
  "stop_condition": "terminal_action",
  "step_count": 12,
  "seed": 18421,
  "fixture_id": "fixture1",
  "final_verifier_verdicts": {
    "VALID_UI": "PASS",
    "NO_CRASH": "PASS",
    "VU-CHECKOUT": "PASS",
    "ROOM_DEVICE_DB": "PASS"
  }
}

Full file: t098-metadata.json.

ATIF step shape

User turn + first agent click:

{
  "schema_version": "ATIF-v1.7",
  "trajectory_id": "vu.checkout.t098__ep_c8c22030d0",
  "steps": [
    {
      "step_id": 1,
      "source": "user",
      "message": "Find me a red {target_query} and checkout."
    },
    {
      "step_id": 2,
      "source": "agent",
      "tool_calls": [
        {
          "function_name": "click",
          "arguments": { "point_2d": [310, 125] }
        }
      ],
      "observation": {
        "results": [
          {
            "content": [
              { "type": "text", "text": "screen_id=HOME; ui_element_ids=[...]" },
              {
                "type": "image",
                "source": { "media_type": "image/jpeg", "path": "screenshots/000000.jpg" }
              }
            ],
            "extra": {
              "screen_id": "HOME",
              "verifiers": [
                { "verifier_id": "VU-CHECKOUT", "status": "IN_PROGRESS" }
              ]
            }
          }
        ]
      }
    }
  ]
}

Excerpt: t098-trajectory-excerpt.json.

Task definition (library row)

{
  "task_id": "vu.checkout.t098",
  "instruction": "Find me a red hoodie and checkout.",
  "verifier_id": "VU-CHECKOUT",
  "target_query": "hoodie",
  "target_sku": "B07HOODIE02",
  "target_variant_name": "Red",
  "constraints": { "clean_cart_required": true, "must_be_in_stock": true },
  "require_needs_confirmation": true,
  "max_steps": 38
}

Other pack outcomes

Trajectory	Outcome	Verifier
`vu.search.add.t202__ep_69493218ce`	PASS (+1)	`VU-CHEAPEST-QTY`
`vu.search.add.t309__ep_b9b6d94734`	PASS (+1)	`VU-SEARCH-ADD`
`vu.checkout.t005__…`	FAIL (−1)	`VU-CHECKOUT`
`vu.search.add.t307__…`	FAIL (−1)	`VU-FILTER-ADD`
`vu.reason.compare.t103__…`	FAIL (−1), truncated	`VU-COMPARE-SWAP`

Pack pass rate: 0.25 (3 / 12), not curated for success. Full narrative: pack README.md.

How this maps to live APIs

Offline pack	Live API
`metadata.terminal_reward`	`GET …/status` → `results[].reward`
`trajectory.json` steps	`GET /dashboard/runs/{run}/rollouts/{id}` → `steps`
`screenshots/*.jpg`	`…/steps/{i}/screenshot`
`verifier_results.jsonl`	Step `verifier_results` / status `verifiers`
`metrics/summary.json`	`/dashboard/stats` + pass@N (fleet-wide, not pack-scoped)

Reproduce live

curl -s -X POST https://agi.akhara.ai/evaluations/run \
  -H 'Content-Type: application/json' \
  -d '{
    "env_id": "amazon_gym_android",
    "task_ids": ["vu.checkout.t098"],
    "rollouts_per_example": 1,
    "endpoint": { "url": "https://your-agent.example.com" }
  }'

Then follow Scores and stats.

Getting started

Mechanics

API reference

Scores and trajectories

Sample trajectories

Pack layout

Highlight: successful checkout (T098)

Metadata

ATIF step shape

Task definition (library row)

Other pack outcomes

How this maps to live APIs

Reproduce live

​Pack layout

​Highlight: successful checkout (T098)

​Metadata

​ATIF step shape

​Task definition (library row)

​Other pack outcomes

​How this maps to live APIs

​Reproduce live

Pack layout

Highlight: successful checkout (T098)

Metadata

ATIF step shape

Task definition (library row)

Other pack outcomes

How this maps to live APIs

Reproduce live