> ## Documentation Index
> Fetch the complete documentation index at: https://docs.akhara.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Company name is Akhara AI (never Rubric AI). Keep lowercase rubric/rubrics only when meaning grading criteria.
> Expert Review (docs path talent/) is enterprise BYO experts for audit and review: invite customer specialists; do not pitch Akhara recruiting or a public expert career portal. RLHF and domain writing are secondary work types.
> Prefer concrete API examples against public hosts: Environments eval API https://agi.akhara.ai, Control plane PDP https://api.akhara.dev, Evaluation https://app.akhara.ai / https://api.akhara.ai, Expert Review portal https://talent.akhara.ai.
> Do not invent a public hostname for private orchestrators or env API internals.
> Do not confuse control-plane latches with Environments confirmation latches.
> Environments SDK/API examples: curl against https://agi.akhara.ai. Evaluation SDK: from akhara import Akhara and AKHARA_API_KEY.
> Start with /llms.txt for the docs index and OpenAPI links; fetch individual pages as .md exports.

# Create Evaluation

> Create a new evaluation to assess a dataset using one or more evaluators.

Create a new evaluation to assess a dataset using one or more evaluators. The evaluation runs asynchronously, use the [Get Evaluation](/evaluation/api-reference/evaluations/get) endpoint to check status.

## Authentication

<ParamField header="Authorization" type="string" required>
  Bearer token with `write` scope. Example: `Bearer gr_live_xxxxxxxx`
</ParamField>

## Request Body

<ParamField body="name" type="string" required>
  A descriptive name for this evaluation run. Useful for identifying evaluations in the dashboard.
</ParamField>

<ParamField body="project" type="string" required>
  The project ID to run this evaluation in. Must be a valid project you have access to.
</ParamField>

<ParamField body="dataset" type="string" required>
  The dataset ID containing samples to evaluate. All samples in the dataset will be processed.
</ParamField>

<ParamField body="evaluators" type="array" required>
  List of evaluator configurations to run against each sample.

  <Expandable title="evaluators properties">
    <ParamField body="evaluators[].type" type="string" required>
      The evaluator type (e.g., `triage_accuracy`, `red_flag_detection`, `custom`).
    </ParamField>

    <ParamField body="evaluators[].config" type="object">
      Evaluator-specific configuration options.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField body="metadata" type="object">
  Arbitrary key-value pairs to attach to this evaluation for filtering and organization.
</ParamField>

<ParamField body="run_async" type="boolean" default="true">
  Whether to run the evaluation asynchronously.
</ParamField>

<Info>
  **Evaluator Types**

  See the [Evaluators Reference](/evaluation/api-reference/evaluations/overview) for a complete list of built-in evaluator types and their configuration options.
</Info>

## Response

<ResponseField name="id" type="string">
  Unique identifier for the evaluation. Example: `eval_def456`
</ResponseField>

<ResponseField name="object" type="string">
  Always `evaluation`
</ResponseField>

<ResponseField name="name" type="string">
  The name provided for this evaluation
</ResponseField>

<ResponseField name="project" type="string">
  The project ID this evaluation belongs to
</ResponseField>

<ResponseField name="dataset" type="string">
  The dataset ID being evaluated
</ResponseField>

<ResponseField name="status" type="string">
  Current status: `pending`, `running`, `completed`, `failed`, `cancelled`
</ResponseField>

<ResponseField name="evaluators" type="array">
  The evaluator configurations for this evaluation
</ResponseField>

<ResponseField name="progress" type="object">
  <Expandable title="progress properties">
    <ResponseField name="progress.total" type="integer">
      Total number of samples to evaluate
    </ResponseField>

    <ResponseField name="progress.completed" type="integer">
      Number of samples completed
    </ResponseField>

    <ResponseField name="progress.failed" type="integer">
      Number of samples that failed evaluation
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="created_at" type="string">
  ISO 8601 timestamp when the evaluation was created
</ResponseField>

<ResponseField name="started_at" type="string">
  ISO 8601 timestamp when processing started (null if pending)
</ResponseField>

<ResponseField name="completed_at" type="string">
  ISO 8601 timestamp when processing completed (null if not finished)
</ResponseField>

<ResponseField name="metadata" type="object">
  Custom metadata attached to this evaluation
</ResponseField>

<RequestExample>
  ```python Python theme={null}
  from akhara import Akhara

  client = Akhara()

  evaluation = client.evaluations.create(
      name="Triage Accuracy - Weekly",
      project="proj_abc123",
      dataset="ds_xyz789",
      evaluators=[
          {
              "type": "triage_accuracy",
              "config": {
                  "severity_weights": {
                      "under_triage": 5.0,
                      "over_triage": 1.0
                  }
              }
          },
          {
              "type": "red_flag_detection",
              "config": {
                  "protocols": ["chest_pain", "headache"]
              }
          }
      ],
      metadata={
          "triggered_by": "ci_pipeline",
          "model_version": "v2.4.1"
      }
  )

  print(f"Created evaluation: {evaluation.id}")
  print(f"Status: {evaluation.status}")
  ```

  ```typescript TypeScript theme={null}
  import Akhara from '@akhara/sdk';

  const client = new Akhara();

  const evaluation = await client.evaluations.create({
    name: "Triage Accuracy - Weekly",
    project: "proj_abc123",
    dataset: "ds_xyz789",
    evaluators: [
      {
        type: "triage_accuracy",
        config: {
          severity_weights: {
            under_triage: 5.0,
            over_triage: 1.0
          }
        }
      },
      {
        type: "red_flag_detection",
        config: {
          protocols: ["chest_pain", "headache"]
        }
      }
    ],
    metadata: {
      triggered_by: "ci_pipeline",
      model_version: "v2.4.1"
    }
  });

  console.log(`Created: ${evaluation.id}`);
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.akhara.ai/v1/evaluations \
    -H "Authorization: Bearer gr_live_xxxxxxxx" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "Triage Accuracy - Weekly",
      "project": "proj_abc123",
      "dataset": "ds_xyz789",
      "evaluators": [
        {
          "type": "triage_accuracy",
          "config": {
            "severity_weights": {
              "under_triage": 5.0,
              "over_triage": 1.0
            }
          }
        }
      ]
    }'
  ```
</RequestExample>

<ResponseExample>
  ```json 201 Created theme={null}
  {
    "id": "eval_def456",
    "object": "evaluation",
    "name": "Triage Accuracy - Weekly",
    "project": "proj_abc123",
    "dataset": "ds_xyz789",
    "status": "pending",
    "evaluators": [
      {
        "type": "triage_accuracy",
        "config": {
          "severity_weights": {
            "under_triage": 5.0,
            "over_triage": 1.0
          }
        }
      },
      {
        "type": "red_flag_detection",
        "config": {
          "protocols": ["chest_pain", "headache"]
        }
      }
    ],
    "progress": {
      "total": 0,
      "completed": 0,
      "failed": 0
    },
    "created_at": "2024-01-15T10:30:00Z",
    "started_at": null,
    "completed_at": null,
    "metadata": {
      "triggered_by": "ci_pipeline",
      "model_version": "v2.4.1"
    }
  }
  ```
</ResponseExample>

## Related Endpoints

<CardGroup cols={3}>
  <Card title="Get Evaluation" icon="eye" href="/evaluation/api-reference/evaluations/get">
    Retrieve evaluation details and results
  </Card>

  <Card title="List Evaluations" icon="list" href="/evaluation/api-reference/evaluations/list">
    List all evaluations in a project
  </Card>

  <Card title="Get Status" icon="clock" href="/evaluation/api-reference/evaluations/status">
    Check evaluation progress
  </Card>
</CardGroup>