> ## Documentation Index
> Fetch the complete documentation index at: https://docs.akhara.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Company name is Akhara AI (never Rubric AI). Keep lowercase rubric/rubrics only when meaning grading criteria.
> Expert Review (docs path talent/) is enterprise BYO experts for audit and review: invite customer specialists; do not pitch Akhara recruiting or a public expert career portal. RLHF and domain writing are secondary work types.
> Prefer concrete API examples against public hosts: Environments eval API https://agi.akhara.ai, Control plane PDP https://api.akhara.dev, Evaluation https://app.akhara.ai / https://api.akhara.ai, Expert Review portal https://talent.akhara.ai.
> Do not invent a public hostname for private orchestrators or env API internals.
> Do not confuse control-plane latches with Environments confirmation latches.
> Environments SDK/API examples: curl against https://agi.akhara.ai. Evaluation SDK: from akhara import Akhara and AKHARA_API_KEY.
> Start with /llms.txt for the docs index and OpenAPI links; fetch individual pages as .md exports.

# Export Results for Regulators

> Generate comprehensive evaluation reports, audit trails, and compliance documentation for FDA submissions, SOC 2 audits, and regulatory inspections.

## Regulatory Requirements Overview

Healthcare AI systems must demonstrate safety and efficacy through documented testing. Different regulatory frameworks have specific requirements:

| Framework            | Applies To                        | Key Requirements                                                     |
| -------------------- | --------------------------------- | -------------------------------------------------------------------- |
| FDA 510(k) / De Novo | Software as Medical Device (SaMD) | Clinical validation, performance testing, risk analysis              |
| FDA AI/ML Guidance   | Adaptive AI systems               | Change management, ongoing monitoring, predetermined change protocol |
| HIPAA                | Any PHI handling                  | Audit logs, access controls, encryption                              |
| SOC 2 Type II        | Enterprise SaaS                   | Security controls, availability, processing integrity                |
| CE Marking (MDR)     | EU medical devices                | Clinical evaluation, post-market surveillance                        |
| ISO 13485            | Medical device QMS                | Design controls, risk management, traceability                       |

## Step 1: Configure Compliance Export

Set up your project for regulatory export with required metadata:

```python title="configure_compliance.py" theme={null}
from akhara import Akhara

client = Akhara(api_key="your-api-key")

# Configure project for regulatory compliance
client.projects.update(
    "patient-triage",
    compliance_config={
        # Regulatory frameworks you're targeting
        "frameworks": ["fda_510k", "hipaa", "soc2"],

        # Device classification (for FDA)
        "device_classification": {
            "product_code": "QAS",  # Clinical decision support
            "device_class": "II",
            "intended_use": "AI-assisted patient triage for non-emergency symptoms"
        },

        # Risk management (ISO 14971)
        "risk_classification": {
            "severity": "serious",  # Could delay necessary care
            "probability": "remote",
            "risk_level": "moderate"
        },

        # Data retention requirements
        "retention": {
            "evaluation_results": "7_years",
            "audit_logs": "7_years",
            "model_artifacts": "lifetime_of_device"
        },

        # Required signatories for exports
        "authorized_signatories": [
            {"name": "Dr. Jane Smith", "title": "Chief Medical Officer", "role": "clinical_validation"},
            {"name": "John Doe", "title": "VP Engineering", "role": "technical_validation"},
            {"name": "Sarah Johnson", "title": "Head of Compliance", "role": "regulatory_review"}
        ]
    }
)
```

## Step 2: Generate FDA Submission Package

Export a complete validation package for FDA 510(k) or De Novo submission:

```python title="fda_export.py" theme={null}
# Generate FDA submission package
fda_package = client.exports.create_fda_package(
    project="patient-triage",

    # Evaluation runs to include
    evaluations=[
        "eval_clinical_validation_v2",
        "eval_safety_testing_final",
        "eval_edge_case_analysis"
    ],

    # Package configuration
    config={
        "submission_type": "510k",  # or "de_novo", "pma"

        # Performance summary
        "performance_summary": {
            "primary_endpoint": "triage_accuracy",
            "secondary_endpoints": ["safety_score", "sensitivity_by_condition"],
            "include_confidence_intervals": True,
            "confidence_level": 0.95
        },

        # Test dataset documentation
        "dataset_documentation": {
            "include_demographics": True,
            "include_source_description": True,
            "include_sampling_methodology": True,
            "include_ground_truth_methodology": True
        },

        # Clinical validation
        "clinical_validation": {
            "include_human_review_results": True,
            "include_reviewer_credentials": True,
            "include_inter_rater_reliability": True
        },

        # Risk analysis
        "risk_analysis": {
            "include_failure_mode_analysis": True,
            "include_hazard_assessment": True,
            "include_mitigation_measures": True
        }
    }
)

print(f"Package ID: {fda_package.id}")
print(f"Status: {fda_package.status}")

# Download when ready
if fda_package.status == "complete":
    client.exports.download(
        fda_package.id,
        destination="./fda_submission_package.zip"
    )
```

<Info>
  **FDA Package Contents**: The FDA package includes: Executive Summary, Performance Testing Report, Clinical Validation Report, Risk Analysis (FMEA), Software Documentation, Labeling/Intended Use, and all supporting data files.
</Info>

## Step 3: Export Performance Testing Report

Generate a detailed performance report with statistical analysis:

```python title="performance_report.py" theme={null}
# Generate standalone performance report
performance_report = client.exports.create_performance_report(
    project="patient-triage",
    evaluation="eval_clinical_validation_v2",

    config={
        "format": "pdf",  # or "docx", "html"

        # Report sections
        "sections": {
            "executive_summary": True,
            "methodology": True,
            "dataset_description": True,
            "results_summary": True,
            "detailed_metrics": True,
            "subgroup_analysis": True,
            "failure_analysis": True,
            "statistical_analysis": True,
            "conclusions": True
        },

        # Metrics to include
        "metrics": {
            "primary": ["accuracy", "sensitivity", "specificity", "ppv", "npv"],
            "by_condition": ["chest_pain", "stroke", "pediatric_fever", "psychiatric"],
            "by_demographic": ["age_group", "sex"],
            "confusion_matrix": True,
            "roc_curves": True,
            "calibration_plots": True
        },

        # Statistical rigor
        "statistics": {
            "confidence_intervals": True,
            "confidence_level": 0.95,
            "method": "wilson",  # For proportions
            "hypothesis_tests": ["mcnemar"],  # vs baseline
            "effect_sizes": True
        },

        # Failure analysis
        "failure_analysis": {
            "include_examples": True,
            "max_examples": 10,
            "redact_phi": True,
            "categorize_by_type": True
        }
    }
)

# Wait for generation
report = client.exports.wait(performance_report.id)
print(f"Report ready: {report.download_url}")
```

Example output structure:

```plaintext title="Performance Report Structure" theme={null}
PERFORMANCE TESTING REPORT
Patient Triage AI System v2.4.1
Generated: 2024-03-15

1. EXECUTIVE SUMMARY
   - Overall Accuracy: 86.2% (95% CI: 83.8-88.4%)
   - Safety Score: 95.4% (95% CI: 93.7-96.8%)
   - Critical Finding: Zero missed life-threatening conditions

2. METHODOLOGY
   - Test Dataset: 2,847 cases (stratified sample)
   - Ground Truth: Board-certified physician consensus
   - Evaluation Protocol: Blinded, randomized review

3. DATASET DESCRIPTION
   - Demographics: 52% female, mean age 47.2 years
   - Condition Distribution: [table]
   - Geographic Distribution: [table]
   - Collection Period: Jan 2023 - Dec 2023

4. RESULTS BY TRIAGE LEVEL
   - Emergent: Sensitivity 99.2%, Specificity 94.1%
   - Urgent: Sensitivity 91.3%, Specificity 88.7%
   - Semi-Urgent: Sensitivity 84.5%, Specificity 91.2%
   - Routine: Sensitivity 88.9%, Specificity 85.3%

5. RESULTS BY CONDITION
   - Cardiac (n=312): Sensitivity 98.4%
   - Neurological (n=198): Sensitivity 99.0%
   - Pediatric (n=245): Sensitivity 95.5%
   [continued...]

6. SUBGROUP ANALYSIS
   - By Age Group: [table with CIs]
   - By Sex: [table with CIs]
   - Disparity Analysis: No statistically significant differences

7. FAILURE ANALYSIS
   - Total Failures: 392/2847 (13.8%)
   - Over-triage: 287 (73.2% of failures)
   - Under-triage: 105 (26.8% of failures)
   - Critical Under-triage: 0 (0.0%)
   [Example cases with redacted PHI]

8. STATISTICAL ANALYSIS
   - Comparison to Baseline v2.3.0: +2.1% accuracy (p=0.003)
   - McNemar's Test: χ² = 8.92, p = 0.003
   - Effect Size (Cohen's h): 0.15 (small)

9. CONCLUSIONS
   [Summary and recommendations]

APPENDICES
A. Confusion Matrices
B. ROC Curves
C. Calibration Plots
D. Complete Failure Case List
E. Reviewer Credentials
```

## Step 4: Export Audit Trail

Generate comprehensive audit logs for SOC 2 or HIPAA compliance:

```python title="audit_export.py" theme={null}
# Export audit trail
audit_export = client.exports.create_audit_trail(
    project="patient-triage",

    # Time range
    start_date="2024-01-01",
    end_date="2024-03-31",

    config={
        "format": "json",  # or "csv", "parquet"

        # Event types to include
        "event_types": [
            "evaluation_created",
            "evaluation_completed",
            "sample_created",
            "sample_reviewed",
            "model_deployed",
            "safety_gate_result",
            "override_requested",
            "override_approved",
            "data_accessed",
            "export_created",
            "settings_changed"
        ],

        # Fields to include
        "fields": [
            "timestamp",
            "event_type",
            "actor_id",
            "actor_email",
            "actor_role",
            "resource_type",
            "resource_id",
            "action",
            "details",
            "ip_address",
            "user_agent",
            "result"
        ],

        # Compliance annotations
        "compliance_annotations": {
            "hipaa_category": True,  # Access, modification, disclosure
            "soc2_control": True     # Map to SOC 2 control objectives
        },

        # PHI handling
        "phi_handling": {
            "redact_patient_data": True,
            "include_access_logs": True,
            "include_phi_flags": True
        }
    }
)

# Download audit trail
client.exports.download(
    audit_export.id,
    destination="./audit_trail_q1_2024.json"
)
```

Example audit log entry:

```json title="Audit Log Entry" theme={null}
{
  "timestamp": "2024-03-15T14:32:18.847Z",
  "event_type": "sample_reviewed",
  "actor_id": "usr_Dr8mKp2n",
  "actor_email": "dr.smith@hospital.org",
  "actor_role": "physician_reviewer",
  "resource_type": "sample",
  "resource_id": "smp_Kj9pL2mN",
  "action": "submit_review",
  "details": {
    "review_type": "clinical_validation",
    "grades_submitted": {
      "triage_accuracy": "correct",
      "safety_flags": "all_addressed"
    },
    "time_spent_seconds": 142,
    "phi_accessed": true
  },
  "ip_address": "10.0.1.45",
  "user_agent": "Mozilla/5.0...",
  "result": "success",
  "hipaa_category": "access",
  "soc2_controls": ["CC6.1", "CC7.2"]
}
```

## Step 5: Export Human Review Documentation

Document the clinical expert review process for validation:

```python title="human_review_export.py" theme={null}
# Export human review documentation
review_docs = client.exports.create_human_review_report(
    project="patient-triage",
    evaluation="eval_clinical_validation_v2",

    config={
        # Reviewer documentation
        "reviewer_documentation": {
            "credentials": True,           # Licenses, certifications
            "credential_verification": True,  # How verified
            "training_records": True,      # Calibration completed
            "conflict_of_interest": True   # COI declarations
        },

        # Review process documentation
        "process_documentation": {
            "review_protocol": True,       # Step-by-step process
            "blinding_methodology": True,  # How reviewers were blinded
            "randomization": True,         # How cases were assigned
            "quality_controls": True       # Gold standard insertion, etc.
        },

        # Inter-rater reliability
        "reliability_analysis": {
            "cohens_kappa": True,
            "fleiss_kappa": True,          # For >2 reviewers
            "percent_agreement": True,
            "by_category": True,
            "confidence_intervals": True
        },

        # Consensus handling
        "consensus_documentation": {
            "disagreement_rate": True,
            "adjudication_process": True,
            "final_consensus_method": True
        },

        # Individual review data (de-identified)
        "review_data": {
            "include_individual_grades": True,
            "include_reviewer_notes": True,
            "redact_phi": True,
            "de_identify_reviewers": False  # Keep for credential linking
        }
    }
)

print(f"Review documentation: {review_docs.download_url}")
```

## Step 6: Generate Reproducibility Manifest

Create a cryptographically signed manifest for exact reproducibility:

```python title="reproducibility_manifest.py" theme={null}
# Generate reproducibility manifest
manifest = client.exports.create_reproducibility_manifest(
    evaluation="eval_clinical_validation_v2",

    config={
        # Include cryptographic hashes
        "hashes": {
            "algorithm": "sha256",
            "include": [
                "model_weights",
                "model_config",
                "evaluation_config",
                "dataset",
                "evaluator_code"
            ]
        },

        # Version information
        "versions": {
            "model_version": True,
            "sdk_version": True,
            "evaluator_versions": True,
            "dependencies": True
        },

        # Environment capture
        "environment": {
            "python_version": True,
            "package_versions": True,
            "system_info": True,
            "gpu_info": True
        },

        # Timestamps
        "timestamps": {
            "evaluation_start": True,
            "evaluation_end": True,
            "data_snapshot": True
        },

        # Digital signature
        "signature": {
            "sign": True,
            "certificate": "projects/patient-triage/certificates/signing-cert"
        }
    }
)

# Verify manifest integrity
verification = client.exports.verify_manifest(manifest.id)
print(f"Signature Valid: {verification.signature_valid}")
print(f"Hashes Match: {verification.all_hashes_match}")
print(f"Reproducible: {verification.is_reproducible}")
```

Example manifest:

```json title="Reproducibility Manifest" theme={null}
{
  "manifest_version": "1.0",
  "evaluation_id": "eval_clinical_validation_v2",
  "created_at": "2024-03-15T10:30:00Z",

  "hashes": {
    "model_weights": "sha256:a3f2e8c9d1b4...",
    "model_config": "sha256:7c8d9e0f1a2b...",
    "evaluation_config": "sha256:4e5f6a7b8c9d...",
    "dataset": "sha256:1a2b3c4d5e6f...",
    "evaluator_triage_accuracy": "sha256:9d8c7b6a5e4f...",
    "evaluator_safety_score": "sha256:3f2e1d0c9b8a..."
  },

  "versions": {
    "model": "v2.4.1",
    "akhara_sdk": "1.2.3",
    "evaluators": {
      "triage_accuracy": "2.1.0",
      "safety_score": "1.5.2"
    }
  },

  "environment": {
    "python": "3.11.4",
    "torch": "2.1.0",
    "transformers": "4.35.0"
  },

  "execution": {
    "start": "2024-03-15T08:00:00Z",
    "end": "2024-03-15T10:28:47Z",
    "samples_processed": 2847,
    "compute_hours": 2.48
  },

  "signature": {
    "algorithm": "ECDSA-SHA256",
    "certificate": "CN=Akhara Clinical Validation",
    "signature": "MEUCIQC7x9Hk..."
  }
}
```

## Step 7: Schedule Regular Exports

Set up automated exports for ongoing compliance:

```python title="scheduled_exports.py" theme={null}
# Configure scheduled exports
client.exports.create_schedule(
    name="monthly-compliance-export",
    project="patient-triage",

    schedule={
        "frequency": "monthly",
        "day_of_month": 1,
        "time": "02:00",
        "timezone": "America/New_York"
    },

    exports=[
        {
            "type": "performance_summary",
            "config": {
                "period": "previous_month",
                "format": "pdf",
                "include_trends": True
            }
        },
        {
            "type": "audit_trail",
            "config": {
                "period": "previous_month",
                "format": "json"
            }
        },
        {
            "type": "safety_gate_summary",
            "config": {
                "period": "previous_month",
                "include_overrides": True
            }
        }
    ],

    delivery={
        "method": "s3",
        "bucket": "compliance-exports",
        "prefix": "patient-triage/monthly/",
        "notify": ["compliance@company.com"]
    }
)

# Also create quarterly QBR export
client.exports.create_schedule(
    name="quarterly-qbr-export",
    project="patient-triage",

    schedule={
        "frequency": "quarterly",
        "day_of_quarter": 5,  # 5th day of quarter
        "time": "02:00"
    },

    exports=[
        {
            "type": "executive_summary",
            "config": {
                "period": "previous_quarter",
                "format": "pptx",
                "include_charts": True,
                "include_recommendations": True
            }
        }
    ]
)
```

## Export Types Summary

| Export Type              | Use Case                          | Format Options     |
| ------------------------ | --------------------------------- | ------------------ |
| FDA Package              | 510(k), De Novo, PMA submissions  | ZIP (structured)   |
| Performance Report       | Clinical validation documentation | PDF, DOCX, HTML    |
| Audit Trail              | SOC 2, HIPAA audits               | JSON, CSV, Parquet |
| Human Review Report      | Reviewer credential documentation | PDF, DOCX          |
| Reproducibility Manifest | Exact reproduction of results     | JSON (signed)      |
| Safety Gate Summary      | Deployment decision documentation | PDF, JSON          |
| Executive Summary        | Board/investor reporting          | PPTX, PDF          |
| Raw Data Export          | Custom analysis, archives         | Parquet, CSV       |

## Compliance Checklist

| Requirement                  | Export Type              | Frequency      |
| ---------------------------- | ------------------------ | -------------- |
| Clinical validation evidence | Performance Report       | Per release    |
| Reviewer credentials on file | Human Review Report      | Quarterly      |
| Access audit logs            | Audit Trail              | Monthly        |
| Change control documentation | Safety Gate Summary      | Per deployment |
| Reproducibility evidence     | Reproducibility Manifest | Per evaluation |
| Post-market surveillance     | Performance Report       | Monthly        |
| Risk management updates      | FDA Package              | Annual minimum |

## Next Steps

* [Evaluation Versioning](/evaluation/docs/evaluation-framework/versioning) - Maintain audit trails across versions
* [Human Review Design](/evaluation/docs/evaluation-framework/human-review-design) - Document reviewer qualifications and workflows
* [Exports API Reference](/evaluation/docs/tutorials/export) - Full API documentation