> ## Documentation Index
> Fetch the complete documentation index at: https://docs.akhara.ai/llms.txt
> Use this file to discover all available pages before exploring further.
> ## Agent Instructions
> Company name is Akhara AI (never Rubric AI). Keep lowercase rubric/rubrics only when meaning grading criteria.
> Expert Review (docs path talent/) is enterprise BYO experts for audit and review: invite customer specialists; do not pitch Akhara recruiting or a public expert career portal. RLHF and domain writing are secondary work types.
> Prefer concrete API examples against public hosts: Environments eval API https://agi.akhara.ai, Control plane PDP https://api.akhara.dev, Evaluation https://app.akhara.ai / https://api.akhara.ai, Expert Review portal https://talent.akhara.ai.
> Do not invent a public hostname for private orchestrators or env API internals.
> Do not confuse control-plane latches with Environments confirmation latches.
> Environments SDK/API examples: curl against https://agi.akhara.ai. Evaluation SDK: from akhara import Akhara and AKHARA_API_KEY.
> Start with /llms.txt for the docs index and OpenAPI links; fetch individual pages as .md exports.
# Human Expert Network
> Akhara's network of credentialed healthcare professionals provides expert review for AI outputs that require clinical judgment.
## Who Reviews
Our expert network includes licensed healthcare professionals across specialties, each verified and matched to appropriate review tasks.
Board-certified MDs and DOs across specialties including emergency medicine, internal medicine, pediatrics, and more.
RNs and APRNs with clinical experience in triage, telehealth, and specialty care.
Certified coders (CPC, CCS) for documentation accuracy and coding validation.
Dietitians, mental health counselors, pharmacists, and other specialists.
## Reviewer Categories
| Category | Credentials | Use Cases |
| ------------------ | ------------------------ | ------------------------------------------------------------------------------------- |
| **Physicians** | MD, DO (Board Certified) | Triage validation, diagnosis review, treatment recommendations, safety-critical cases |
| **Nurses** | RN, BSN, MSN, NP | Triage screening, symptom assessment, patient education review |
| **Medical Coders** | CPC, CCS, RHIA, RHIT | ICD-10 validation, CPT accuracy, documentation completeness |
| **Dietitians** | RD, RDN, LD | Nutrition advice, dietary recommendations, meal planning AI |
| **Mental Health** | LCSW, LPC, Psychologist | Crisis assessment, therapy recommendations, mental health triage |
| **Pharmacists** | PharmD, RPh | Medication interactions, dosing validation, drug information |
## Credentialing & Verification
Every reviewer in our network undergoes rigorous credentialing before accessing any review tasks.
### Verification Process
```
1. Application & Documentation
├── License verification (primary source)
├── Board certification check
├── Education verification
└── Background check
2. Skills Assessment
├── Clinical knowledge test
├── Calibration case review
└── Platform training
3. Ongoing Monitoring
├── License status monitoring
├── Performance tracking
└── Annual re-credentialing
```
**Primary Source Verification**: All licenses are verified directly with state medical boards and credentialing bodies, never relying solely on self-reported information.
### Credential Tracking
```python theme={null}
from akhara import Akhara
client = Akhara()
# View reviewer credentials for your evaluation
reviewers = client.reviewers.list(
evaluation_id="eval_abc123",
include_credentials=True
)
for reviewer in reviewers:
print(f"""
Reviewer: {reviewer.id}
Type: {reviewer.credential_type} # e.g., "physician"
Specialty: {reviewer.specialty}
License State: {reviewer.license_state}
License Status: {reviewer.license_status}
License Verified: {reviewer.license_verified_at}
Board Certified: {reviewer.board_certified}
Platform Stats:
Reviews Completed: {reviewer.total_reviews}
Agreement Rate: {reviewer.agreement_rate:.1%}
Avg Review Time: {reviewer.avg_review_time_seconds:.0f}s
""")
```
## Reviewer Assignment Logic
Cases are matched to reviewers based on clinical requirements, expertise, and availability.
```python title="assignment_config.py" theme={null}
assignment_rules = {
"matching_criteria": {
# Required credentials
"credential_requirements": {
"emergency_triage": ["physician", "emergency_nurse"],
"routine_triage": ["nurse", "physician"],
"mental_health": ["mental_health_professional"],
"pediatric": ["pediatric_specialist", "pediatric_nurse"],
"coding_review": ["certified_coder"]
},
# Specialty matching
"specialty_matching": {
"cardiology_cases": ["cardiology", "internal_medicine", "emergency"],
"oncology_cases": ["oncology", "hematology"],
"obstetric_cases": ["obstetrics", "maternal_fetal"]
},
# Experience requirements
"experience_thresholds": {
"safety_critical": {"min_reviews": 100, "min_accuracy": 0.95},
"training_data": {"min_reviews": 50, "min_accuracy": 0.90},
"standard": {"min_reviews": 10, "min_accuracy": 0.85}
}
},
"load_balancing": {
"max_concurrent_assignments": 10,
"max_daily_reviews": 50,
"fatigue_cooldown_hours": 8,
"specialty_rotation": True # Prevent monotony
},
"priority_factors": {
"expertise_match": 0.4,
"availability": 0.3,
"historical_accuracy": 0.2,
"response_time": 0.1
}
}
```
### Assignment API
```python theme={null}
# Configure custom assignment rules
client.evaluations.configure_assignment(
evaluation_id="eval_abc123",
assignment={
"mode": "auto", # auto, manual, hybrid
# Require specific credentials
"require_credentials": ["physician"],
# Prefer certain specialties
"prefer_specialties": ["emergency_medicine", "internal_medicine"],
# Exclude reviewers with conflicts
"exclude_organizations": ["competitor_health_system"],
# Geographic requirements (for state-specific regulations)
"require_state_license": ["CA", "NY", "TX"]
}
)
```
## Conflict-of-Interest Controls
Maintaining objectivity requires systematic conflict-of-interest management.
| Control | Implementation |
| ------------------------- | --------------------------------------------------------- |
| Organizational separation | Reviewers cannot evaluate AI from their employer |
| Competitive exclusion | Option to exclude reviewers from competitor organizations |
| Random assignment | Reviewers cannot select specific cases |
| Blinding | AI source/version hidden from reviewers when appropriate |
| Financial disclosure | Annual disclosure of relevant financial interests |
| Rotating assignments | Prevent patterns that could enable gaming |
```python theme={null}
# Configure conflict-of-interest rules
client.organizations.configure_coi(
# Organizations that cannot review your AI
excluded_organizations=[
"my_company", # Internal employees
"competitor_a",
"competitor_b"
],
# Require conflict disclosure
require_disclosure=True,
# Blind reviewers to AI source
blind_ai_source=True,
# Prevent same reviewer from seeing related cases
prevent_case_clustering=True
)
```
## Quality Assurance & Calibration
### Calibration Program
All reviewers complete initial calibration and ongoing recalibration to ensure consistency.
```python title="calibration_program.py" theme={null}
calibration_config = {
"initial_calibration": {
"required": True,
"gold_standard_cases": 20,
"passing_score": 0.85,
"feedback_provided": True,
"unlimited_attempts": False,
"max_attempts": 3
},
"ongoing_calibration": {
"frequency": "monthly",
"cases_per_session": 10,
"passing_score": 0.80,
"remediation_threshold": 0.70,
"suspension_threshold": 0.60
},
"gold_standard_insertion": {
"enabled": True,
"rate": 0.05, # 5% of cases are gold standards
"flag_poor_performance": True
},
"inter_rater_reliability": {
"measure": "cohens_kappa",
"target": 0.8,
"alert_threshold": 0.6
}
}
```
### Performance Monitoring
```python theme={null}
# Monitor reviewer performance
qa_dashboard = client.reviewers.quality_dashboard()
print(f"""
Network Quality Metrics
=======================
Active Reviewers: {qa_dashboard.active_reviewers}
Avg Inter-rater Reliability (κ): {qa_dashboard.avg_kappa:.2f}
Avg Agreement Rate: {qa_dashboard.avg_agreement:.1%}
Performance Distribution:
Excellent (>95%): {qa_dashboard.excellent_count} reviewers
Good (85-95%): {qa_dashboard.good_count} reviewers
Needs Improvement (70-85%): {qa_dashboard.needs_improvement_count} reviewers
Under Review (<70%): {qa_dashboard.under_review_count} reviewers
Recent Alerts:
""")
for alert in qa_dashboard.recent_alerts:
print(f" ⚠️ {alert.reviewer_id}: {alert.reason}")
```
## Reviewer Specialties
### Emergency Medicine
Board-certified emergency physicians and emergency nurses for high-acuity triage review.
| Credential | Scope |
| ------------------------- | ----------------------------------------------------- |
| Emergency Medicine (ABEM) | All emergency triage, trauma, resuscitation decisions |
| Emergency Nurse (CEN) | Triage screening, ESI level validation |
| Pediatric Emergency (PEM) | Pediatric emergency cases |
### Mental Health
Licensed mental health professionals for crisis assessment and therapy AI review.
| Credential | Scope |
| ------------------------------- | -------------------------------------------------- |
| Psychiatrist (MD) | Crisis assessment, medication recommendations |
| Psychologist (PhD/PsyD) | Therapy recommendations, assessment interpretation |
| Licensed Clinical Social Worker | Crisis screening, resource recommendations |
| Licensed Professional Counselor | General mental health triage |
### Coding & Documentation
Certified coders for clinical documentation and coding accuracy review.
| Credential | Scope |
| ---------------------------------- | ---------------------------------------------- |
| CPC (Certified Professional Coder) | E\&M coding, procedure codes |
| CCS (Certified Coding Specialist) | Inpatient coding, DRG validation |
| RHIA/RHIT | Documentation completeness, health information |
## Bring Your Own Reviewers
You can also use your own clinical staff as reviewers within Akhara.
```python theme={null}
# Register internal reviewers
internal_reviewer = client.reviewers.register(
email="dr.smith@myorg.com",
credentials={
"type": "physician",
"specialty": "internal_medicine",
"license_number": "A123456",
"license_state": "CA",
"npi": "1234567890"
},
# Organization context
organization="my_org",
internal=True,
# Skip external verification (you vouch for them)
skip_verification=True,
# Restrict to your evaluations only
restrict_to_org=True
)
# Create internal-only evaluation
evaluation = client.evaluations.create(
name="Internal QA Review",
dataset="ds_production",
evaluators=[...],
human_review={
"enabled": True,
"reviewer_source": "internal_only", # Only use your reviewers
"reviewer_pool": "my_org"
}
)
```
**Internal Reviewer Responsibility**: When using internal reviewers with skip\_verification, your organization is responsible for verifying credentials and maintaining compliance.