vac:bi:rag:2025q4-rag-evaluation
Description
Setting up an evaluation process for the RAG responses of predefined questions. This will help to determine the gain or regression from different changes in the pipelines.
Task List
Setup an evaluation process
- fully qualified name:
vac:bi:rag:2025q4-rag-evaluation:setup-evaluation-process - owner: nickninov
- status: in progress (40%)
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
https://github.com/status-im/data-docs/issues/99
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- Monitoring data transformed for easier evaluation tracking.
- List of evaluation questions drafted.
- Added a chunks coverage dashboard so evaluators can trace which data sources fed each score.