vac:bi:rag:2025q3-rag-evaluation-pipeline
Description
Setting up a evaluation process for the RAG responses of predefined questions. This will help to determind the gain or regretion from different change in the pipelines.
Task List
Setup an evaluation process
- fully qualified name:
vac:bi:rag:2025q4-rag-evaluation-pipeline - owner: nickninov
- status: todo
- start-date:
- end-date:
Description
https://github.com/status-im/data-docs/issues/99
Deliverables
- List of questions to use as evaluation process
- Pipeline to test the response
- Evaluation process: can be manual or automatic