AISAOCMS

R3 Retrieval Quality Baseline — Implementation Plan (Phase 1)

Add real IR metrics (precision@k, recall@k, nDCG@k) to the application-content evaluation harness and give both registered sources (cms.contents, sao.tickets) a committed, deterministic ranking-quality baseline.

State
Done
Progress
100%
Steps
17 / 17
Started
2026-09-17
Finished
2026-09-17
Modules
LaraAI, LaraSAO, LaraCMS

Tasks.

  • Task 1: IR metrics in ApplicationContentEvaluationService5/5
  • Task 2: Regenerate the CMS deterministic baseline4/4
  • Task 3: SAO curated dataset + deterministic baseline (greenfield)4/4
  • Task 4: Update AI service/command tests to assert @k4/4

Read the full plan on GitHub