vac:dst:codex:2025q4-codex-evaluation
Description
Test Codex on each new version or requested feature and look for regressions if this is required. Help Codex with testing and benchmarking new features.
Background
We want to learn specific, actionable information about Codex’s behaviour and how it is evolving over time with each new release and with each thing we are specifically asked to check and test.
We will use a combination of real world testing, theoretical analysis and experiments.
Narratives
We will support the Conduit of Expertise narrative directly by analysing and evaluating new Codex releases and their features, both with regards to features they have today and with regards to how that compares to past behaviour.
We will:
- Enable improvements to Codex by allowing for repeatable, measureable and real world insights into Codex, all the way from theory to practice and back.
- Reduce the risk of a Codex regression making it into a new release of Codex.
Additionally, these efforts will contribute to the Premier Research destination narrative by:
-
Improving and strengthening our relationship with the Codex team and improving the quality and reputation of IFT’s work, inside and outside of Codex.
Background
Narratives
We will support the Conduit of Expertise narrative directly by providing valuable insights to Codex that allow them to understand how Codex performs in comparison to common and popular systems in the “altruistic” space.
Task List
Codex-in-Status
- fully qualified name:
vac:dst:codex:2025q4-codex-evaluation:codex-in-status - owner: Alberto
- status: not started
- start-date: 2025/11/03
- end-date: 2025/11/15
Description
Assist the Codex team to check the functionality of replacing bittorrent usage in status go (in the Community History Service). Evaluate if the new feature is working as expected by trying to assert:
- Presence of archive in filesytem
- Presence of messages in status DB
Deliverables
- PRs/Issues/Docs/Reports