vac:dst:status:2025q4-status-evaluation
Description
Realizing chat protocol benchmarks on status-go will allow the Status and Waku team to perform non-regression performance tests for chat protocols. Also, this will allow quantify improvements tackled by Waku Chat team. The benchmarks will be done for communities, with emphasis on subscription and store performance, and private chats, with emphasis on contact requests, 1-1 messages and group messages.
Background
The following scenarios will help review what kind of improvements and changes can be made to the chat protocols, and see if more baseline benchmarks are needed to be measured in FURPS.
Task list
Send one-to-one outage (private chats)
- fully qualified name:
vac:dst:status:2025q4-status-evaluation:one-to-one-outage - owner: Alberto
- status: not started
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Notion Link Testing app behaviour when something goes wrong. Best to start with what is easier within test framework: cut nodes from network, stop and start nodes, suspend processes, etc.
Setup:
- 50 sending nodes
- 50 receiving nodes
- 50 idle nodes
- Mix of relay and light nodes, results should be per node type.
- Each sending node has sent a contact request to a receiving node, that accepted it
Test:
- Sending nodes send one text message per 10 sec
- Test runs for 2 hours or so.
- Sending and receiving nodes lose access to network or get killed, restarted
- Duration of “outage” (whatever is the chose form), should go from 1 seconds to 1 hour; Probably a global setting so that we can look at bandwidth usage in relation to outage. and see if specific duration lead to an explosion of messages; or if specific duration lead to very poor UX (e.g. outage 30min, messages then take 60min to eventually arrive due to poor backoff strategy)
- Metrics to ensure all messages are eventually received (100%) is important
- and metrics as specified in this page.
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- PRs/Issues/Docs/Reports
Status-backend private chats - send group outage
- fully qualified name:
vac:dst:status:2025q4-status-evaluation:group-outage - owner: Alberto
- status: not started
- start-date: 2025/10/01
- end-date: 2025/12/31
Description
Notion Link Same as @Send one-to-one message - Network outage for private group
Schedule note: Dates reflect quarter bounds; update when actual timing is known.
Deliverables
- PRs/Issues/Docs/Reports
Chat protocol benchmarks followup
- fully qualified name:
vac:dst:status:2025q4-status-evaluation:chat-protocol-benchmarks-followup - owner: Alberto
- status: in progress (70%)
- start-date: 2025/10/06
- end-date: 2025/10/24
Description
Using the fix in the discovery process for the relay nodes to repeat the benchmarks adding light nodes in the same chat protocol benchmarks scenarios.