Building a Reliable, Timezone-Aware Push Notification System
A health & wellness app needed to send personalized daily nudges — meal reminders, mood check-ins, hydration prompts, sleep cues, and custom reminders — to thousands of users across the globe. The catch: every notification had to arrive at the right local time, exactly once, and never to a dead device. We designed and built the distributed pipeline that makes that happen.
The Challenge
- Timezone correctness at scale. A "9 AM breakfast reminder" means something different for every user. Sending in server time would ping someone in Sydney at 3 AM. Each notification needed to resolve to the user's local moment.
- No spam, no duplication. Scheduling crons inevitably overlap and re-run. Without strict guarantees, a single user could receive the same "time for lunch 🥗" nudge two or three times — a fast track to an uninstall.
- Devices are unreliable. Users uninstall apps, revoke permissions, and rotate push tokens constantly. Firing notifications blindly at stale tokens wastes resources and corrupts delivery metrics.
- Precise timing without a brute-force scheduler. Delivering hundreds of notifications at exact minutes — without a cron hammering the database every 60 seconds — required a smarter mechanism than naive polling.
Our Solution
We built a three-stage pipeline that cleanly separates what to send, when to send it, and actually sending it — so each stage can fail and recover independently. The database is the source of truth, a message queue handles precise timing, and a single worker layer talks to the push provider.
Architecture
- Expo-notifications is a React Native client with native channels, unique noises, and deep links that uses a single token format and delivery API for both iOS and Android.
- NestJS backend with expo-server-sdk as the unified push abstraction over FCM and APNs.
- MongoDB as the source of truth — NotificationMessage, NotificationToken, and NotificationCounter collections.
- ActiveMQ (STOMP) delay queues, one per category (meal, mood, activity, safety, reminders), for precise scheduled delivery.
- Creator crons that generate timezone-resolved notification records per user.
- Consumer workers that subscribe to each queue and perform final validation before sending.
- AWS ECS Fargate running crons and consumers; ActiveMQ on a dedicated EC2 instance.
Key Features
- Timezone-aware scheduling. Using date-fns-tz, each user's local send time is computed, converted back to UTC for storage, and bounded by UTC date windows to guarantee one nudge per day.
- Database-enforced idempotency. A partial unique index on pending messages makes duplicate creation impossible — even when a cron runs twice:
| // Unique only while the message is still PENDING and not deleted schema.index( { userId: 1, notificationTokenId: 1, category: 1, label: 1, scheduledAt: 1 }, { unique: true, partialFilterExpression: { status: 'pending', isDeleted: false } } ); |
3. Precise delivery via delay queues. Instead of a per-minute cron, the scheduler enqueues messages now but defers delivery to the exact due minute using ActiveMQ's scheduled-delay header:
| client.send(`/queue/${queueName}`, { persistent: 'true', 'AMQ_SCHEDULED_DELAY': String(delayMs), // delivered exactly when due }, JSON.stringify(message)); |
4. Single active token per device. A partial unique index guarantees exactly one active token per device; new logins cleanly retire the old token, with exponential-backoff retries to survive concurrent sign-ins.
5. Receipt checking + auto-cleanup. After sending, we poll Expo receipts. A DeviceNotRegistered response immediately deactivates the dead token so we never waste a send on it again.
| f (receipt.status === 'error' && receipt.details?.error === 'DeviceNotRegistered') { await this.deactivateToken(token); // stop sending to dead devices } |
6. Failure ceiling. Each device has a retry counter; after 3 consecutive failures the token is automatically retired — no infinite loops, no zombie tokens.
7. Respecting user intent at send time. Notification preference is re-checked by the consumer at delivery, not just at scheduling — so a user who opts out an hour before a nudge never receives it. Messages resolve to honest terminal states: success, failed, or is_missed.
Results
- Every user receives nudges at the correct local time, worldwide — zero off-hours notifications.
- Duplicate notifications eliminated entirely through database-level idempotency.
- Dead and stale device tokens are detected and retired automatically, keeping delivery clean.
- Reliability is increased and database load is reduced with precise, minute-accurate delivery without a brute-force scheduler.
Technology Stack
React Native · Expo Notifications · NestJS · TypeScript · MongoDB · ActiveMQ (STOMP) · expo-server-sdk · date-fns-tz · AWS ECS Fargate
