> ## Documentation Index
> Fetch the complete documentation index at: https://docs.akhara.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Company name is Akhara AI (never Rubric AI). Keep lowercase rubric/rubrics only when meaning grading criteria.
> Expert Review (docs path talent/) is enterprise BYO experts for audit and review: invite customer specialists; do not pitch Akhara recruiting or a public expert career portal. RLHF and domain writing are secondary work types.
> Prefer concrete API examples against public hosts: Environments eval API https://agi.akhara.ai, Control plane PDP https://api.akhara.dev, Evaluation https://app.akhara.ai / https://api.akhara.ai, Expert Review portal https://talent.akhara.ai.
> Do not invent a public hostname for private orchestrators or env API internals.
> Do not confuse control-plane latches with Environments confirmation latches.
> Environments SDK/API examples: curl against https://agi.akhara.ai. Evaluation SDK: from akhara import Akhara and AKHARA_API_KEY.
> Start with /llms.txt for the docs index and OpenAPI links; fetch individual pages as .md exports.

# Reviewing agent runs

> How invited specialists audit agent trajectories, scores, failures, and policy-sensitive cases, and turn findings into better tasks and rubrics.

Audit and review is the core Expert Review work type. Your organization points its specialists at agent runs from [Environments](/environments) and scored results from [Evaluation](/evaluation/docs/introduction), and you judge whether the agent, the score, and the task itself hold up under expert scrutiny.

## What you review

| Item                       | What you check                                                                                                                                      |
| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Trajectories**           | Step-by-step agent runs: did each action make sense for the goal, and did the agent follow required procedure (confirmations, escalations, checks)? |
| **Scores**                 | Whether the assigned reward or grade matches what actually happened in the run                                                                      |
| **Failures**               | Why a run failed: agent error, ambiguous task, broken verifier, or environment issue                                                                |
| **Policy-sensitive cases** | Runs flagged for safety, privacy, or domain-policy concerns that need a specialist's call                                                           |
| **Task quality**           | Whether the task and its success criteria are unambiguous, checkable, and realistic for the domain                                                  |

## How to review a run

<Steps>
  <Step title="Read the brief first">
    Every queue has a project brief with the rubric and scope. Review against that rubric, not personal preference.
  </Step>

  <Step title="Walk the trajectory">
    Follow the run step by step against the task goal. Note where the agent diverged, skipped a required check, or acted on invented facts.
  </Step>

  <Step title="Check the score against the evidence">
    Confirm the verifier outcome and reward match the run. A passing score on a run that violated domain procedure is a finding.
  </Step>

  <Step title="Classify the failure">
    Separate agent mistakes from task or verifier defects. "The task was ambiguous" and "the agent was wrong" lead to different fixes.
  </Step>

  <Step title="Write the finding">
    Record your judgment with a short rationale tied to specific steps in the run. Flag anything out of your specialty for reassignment instead of guessing.
  </Step>
</Steps>

## What a good finding looks like

* Names the decisive step or span: "step 7 placed the order without the required confirmation turn"
* States the domain rule it violates or satisfies, in terms another specialist can verify
* Distinguishes agent error from task or scoring error
* Recommends a concrete fix when the defect is in the task or rubric: tighten a success criterion, add a failure mode, split an ambiguous goal

Vague findings ("seemed off", "score feels harsh") do not move tasks or rubrics. Falsifiable ones do.

## Where findings go

Review findings feed back into the rest of the suite:

* Task and success-criteria fixes land in your org's [Environments](/environments) task banks
* Rubric corrections and adjudicated hard cases tighten [Evaluation](/evaluation/docs/introduction) datasets and human-review workflows
* Confirmed failures become regression cases so the same defect is caught automatically next time

## Related

* [RLHF evaluations](/talent/guidance/rlhf-evaluations): judging model outputs for preference and safety signal
* [Domain writing for RL environments](/talent/guidance/domain-writing): authoring the tasks agents are graded on
* [Policies](/talent/policies/overview): judgment must be your own; no external LLMs on task content