161 lines
5.2 KiB
Markdown
161 lines
5.2 KiB
Markdown
---
|
|
title: Notification Assertion Procedure
|
|
tags: [testing, notifications, e2e, socket-io]
|
|
created: 2026-06-06
|
|
---
|
|
|
|
# Notification Assertion Procedure
|
|
|
|
Every E2E business step must verify notifications. A test step is incomplete
|
|
until notification persistence and, where practical, real-time delivery are
|
|
validated.
|
|
|
|
Related docs:
|
|
|
|
- [[04 - Flows/Notification Flow]]
|
|
- [[03 - API Reference/Notification API]]
|
|
- [[Escrow Marketplace E2E Procedure]]
|
|
|
|
## Principle
|
|
|
|
For each state-changing action, assert notifications for the expected recipients.
|
|
If the product currently does not emit a notification for that action, the E2E
|
|
must record a notification coverage gap. Silent missing notifications should not
|
|
be treated as a pass.
|
|
|
|
## Notification Channels To Check
|
|
|
|
| Channel | Required? | How |
|
|
|---|---|---|
|
|
| Persistence | Always | `GET /api/notifications`, `GET /api/notifications/unread-count` |
|
|
| Socket push | Required when runner has sockets enabled | Listen for `new-notification` on `user-<userId>` |
|
|
| Badge count sync | Required for read/mark-read steps | `unread-count-update` socket event or count endpoint |
|
|
| Email/push | Not required today | Planned digest/push work is not implemented |
|
|
|
|
## Baseline Before a Scenario
|
|
|
|
For every actor:
|
|
|
|
1. Authenticate.
|
|
2. Read unread count:
|
|
```http
|
|
GET /api/notifications/unread-count
|
|
```
|
|
3. Read latest notifications:
|
|
```http
|
|
GET /api/notifications?page=1&limit=5
|
|
```
|
|
4. Store:
|
|
- `baselineUnreadCount`;
|
|
- latest notification id;
|
|
- timestamp.
|
|
|
|
Do this before creating purchase requests or offers so later assertions can
|
|
distinguish new notifications from old ones.
|
|
|
|
## Per-Step Assertion Algorithm
|
|
|
|
After each action:
|
|
|
|
1. Identify expected recipients.
|
|
2. Poll each recipient's notifications for up to the notification SLA window.
|
|
3. Assert at least one new notification matching the action.
|
|
4. Assert unread count increased by the expected number, unless the notification
|
|
is intentionally auto-read.
|
|
5. Assert `actionUrl` or `relatedId` points to the correct request/payment/offer
|
|
where the notification type supports it.
|
|
6. If socket instrumentation is active, assert the same notification arrived via
|
|
`new-notification`.
|
|
7. Record `notificationLatencyMs = notification.createdAt - actionCompletedAt`
|
|
or runner-observed first-seen time.
|
|
|
|
Suggested polling:
|
|
|
|
| Target | Value |
|
|
|---|---:|
|
|
| poll interval | `500 ms` |
|
|
| timeout | `10 s` for local/dev API notifications |
|
|
| hard failure threshold | `30 s` |
|
|
|
|
## Expected Marketplace Notifications
|
|
|
|
| Action | Expected recipients | Expected category/type |
|
|
|---|---|---|
|
|
| Buyer creates request | eligible sellers | `purchase_request` / new request or opportunity |
|
|
| Seller submits offer | buyer | `offer` / new offer |
|
|
| Buyer accepts offer | selected seller | `offer` / accepted |
|
|
| Buyer accepts offer | non-selected sellers | `offer` / rejected or no-longer-selected, when implemented |
|
|
| Payment intent created | buyer | `payment` / pending or started, when implemented |
|
|
| Scanner confirms payment | buyer, selected seller | `payment` / confirmed or funded |
|
|
| Seller marks delivery | buyer | `delivery` / delivery submitted |
|
|
| Buyer confirms delivery | selected seller | `delivery` / delivery confirmed |
|
|
| Dispute raised | buyer, seller, admin/mediator | `delivery`/`payment`/`system` dispute hold, when implemented |
|
|
| Release/refund | buyer, seller | `payment` / release or refund |
|
|
|
|
Known current gaps from existing docs:
|
|
|
|
- `pending_payment` and `seller_paid` status changes are not covered by
|
|
`NotificationService.notifyRequestStatusChanged`.
|
|
- Dispute service notification emits are TODO stubs in the dashboard dispute path.
|
|
|
|
These known gaps should still be reported by the E2E runner as gaps, not ignored.
|
|
|
|
## Evidence Format
|
|
|
|
For each step, append a notification assertion record:
|
|
|
|
```json
|
|
{
|
|
"step": "seller_offer_created",
|
|
"recipient": "buyer",
|
|
"recipientUserId": "<id>",
|
|
"expected": true,
|
|
"observed": true,
|
|
"latencyMs": 742,
|
|
"unreadBefore": 3,
|
|
"unreadAfter": 4,
|
|
"notificationId": "<id>",
|
|
"category": "offer",
|
|
"actionUrl": "/dashboard/request/<requestId>",
|
|
"socketObserved": true,
|
|
"gap": null
|
|
}
|
|
```
|
|
|
|
For a known gap:
|
|
|
|
```json
|
|
{
|
|
"step": "payment_intent_created",
|
|
"recipient": "buyer",
|
|
"expected": true,
|
|
"observed": false,
|
|
"gap": "No notification emitted for pending_payment status",
|
|
"linkedDoc": "Notification Flow#Purchase request status coverage gap"
|
|
}
|
|
```
|
|
|
|
## Pass/Fail Rules
|
|
|
|
| Condition | Result |
|
|
|---|---|
|
|
| Expected notification appears within SLA | pass |
|
|
| Expected notification appears after SLA but before hard timeout | warning |
|
|
| Expected notification never appears and no approved gap exists | fail |
|
|
| Notification appears for wrong user | fail |
|
|
| Notification exists but has wrong request/payment/offer id | fail |
|
|
| Socket missing but persistence exists | warning unless socket coverage is the target |
|
|
| Unread count inconsistent with persisted notification | fail |
|
|
|
|
## Performance Metrics
|
|
|
|
Notification metrics must be included in concurrency profiles:
|
|
|
|
- notification persistence latency p50/p95/p99;
|
|
- socket delivery latency p50/p95/p99;
|
|
- notification failures by action and recipient;
|
|
- unread count mismatch rate;
|
|
- duplicate notification rate;
|
|
- Mongo notification insert error rate.
|
|
|