AICMS

Assistant End-to-End Evaluation (R1b)

Give the composed in-app assistant a per-module, deterministic evaluation "report card": a dataset of cases run through the real InAppAssistanceService::respond() with a scripted router, scoring composition-plumbing metrics behind a CI regression gate.

State
Done
Progress
100%
Steps
28 / 28
Started
2026-09-15
Finished
2026-09-15
Modules
LaraAI, LaraCMS

Tasks.

  • Task 1: AssistantEvaluationCase value object5/5
  • Task 2: AssistantEvaluationDataset loader5/5
  • Task 3: AssistantEvaluationOutcome reader + AssistantEvaluationService scoring5/5
  • Task 4: Scripted-runner fixture + surface offering assertion5/5
  • Task 5: First-module (CMS) assistant dataset + baseline gate4/4
  • Task 6: RAG documentation2/2
  • Final verification2/2

Read the full plan on GitHub