Running

There are two ways an agent “runs” under Akhara: offline evals (batch, graded by verifiers) and live runtime enforcement (the PEP gating a real session).

Offline eval runs

# Run a whole suite against an agent
akhara eval tasks.py --agent checkout-agent

# Run a single task
akhara eval run --task vu.checkout.hard.t131

Each run produces a trace under runs/<slug>/ with a summary.json (akhara.harbor.trace_export.v1) plus per-step screenshots and verifier results. The index (runs/index.json) records:

{
  "id": "…",
  "slug": "cvs_t131_…",
  "task_id": "vu.checkout.hard.t131",
  "env_id": "env_cvs_gym",
  "agent": "checkout-agent",
  "final_score": 1.0,
  "status": "PASS",
  "step_count": 19,
  "verdicts": { "VALID_UI": "PASS", "VU-CHECKOUT": "PASS" }
}

A step captures the action, what the agent saw, and how each verifier graded it:

{
  "turn": 4,
  "step_index": 4,
  "action_type": "click",
  "action_params": { "point_2d": [630, 578] },
  "screen_id": "SEARCH_RESULTS",
  "screenshot_path": "screenshots/step_04.png",
  "step_reward": 0.04,
  "verifier_results": [{ "id": "VALID_UI", "verdict": "PASS" }]
}

Latch steps are flagged in the trace, so you can see exactly where a policy would have stopped or escalated the agent during evaluation.

Live runtime enforcement

In production the PEP wraps the five stages around a real turn.

A full chat turn

// 1. Gate the incoming request
if (!(await pep.checkInput(userText)).mayContinue) return refuse();

// 2. Minimize sensitive data before context leaves for the LLM
const ctx = await pep.checkContextEgress(accountContextJson);
const modelInput = ctx.transformedContent ?? accountContextJson;   // WARN rewrites here

// 3. Gate the model's draft
const out = await pep.checkOutput(draft);
if (!out.mayContinue) return escalateOrDrop(out);

// 4. Final delivery gate
const delivered = await pep.checkDelivery(out.transformedContent ?? draft);
if (!delivered.mayContinue) return escalateOrDrop(delivered);
return show(delivered);

A consequential action

The permit pattern: the side-effecting service refuses to run without a fresh permit.

const permit = await pep.authorizeAction("refund_payment", { orderId, amount });

if (permit.verdict === "ALLOW") {
  await paymentService.submit(orderId, amount, permit.permitId!); // permit required
} else {
  // BLOCK    → tell the customer it can't be done here
  // ESCALATE → hand off to a supervisor queue
}

Everything is evidence

Every authorize call (allow, warn, block, escalate) is written to the enforcement feed as a policy_decision event:

{
  "ts": 0,
  "type": "policy_decision",
  "data": {
    "agentId": "support-ai",
    "surface": "Support Copilot",
    "verdict": "BLOCK",
    "stage": "output",
    "rule": "PCI DSS Requirement 3.3",
    "policyId": "pci-0",
    "policyMatch": "Unmasked PAN in output",
    "original": "…", "final": "…",
    "delivery": "Response or action withheld",
    "evidence": "Logged",
    "permitId": null,
    "attachedPolicyIds": ["latch-0","…","pci-0"]
  }
}

That feed is your audit trail: nothing an agent does leaves without a record. Decisions can be sealed and later checked with the verification-record endpoints. For the full model (audit trail, signed records, trajectories), see Audit and evidence.

SDK requirements

Runtimes, versions, network, and configuration the PEP needs.

Get started

How it works

Enterprise onboarding

Cookbooks

SDK

Offline eval runs

Live runtime enforcement

A full chat turn

A consequential action

Everything is evidence

SDK requirements

​Offline eval runs

​Live runtime enforcement

​A full chat turn

​A consequential action

​Everything is evidence

SDK requirements

Offline eval runs

Live runtime enforcement

A full chat turn

A consequential action

Everything is evidence