A single backend was being outgrown by an expanding health and nutrition business.AI chatbot traffic, heavy wearable-data ingestion, and everyday API requests were all competing for the same resources — making the system hard to scale and risky to deploy. We re-architected it into focused, independently deployable services orchestrated behind a single API gateway, running on AWS.
The Challenge
- One codebase handling all tasks. One workflow included user management, AI chatbot inference, recipes, and health data analytics. A spike in any one area degraded the entire platform, and every change meant redeploying the whole thing.
- Extremely disparate resource profiles. LLM-powered chatbot requests are CPU- and memory-hungry and bursty; wearable-data ingestion is write-heavy and continuous; core CRUD APIs are light and constant. Sizing one server for all three meant overpaying for some workloads and starving others.
- Independent scaling and deployment. The team needed to scale the AI workload without touching the core API, and ship changes to one domain without risking the others.
A single, safe entrance point. Despite multiple backend services, clients (mobile, web, admin) needed one consistent, authenticated surface to talk to — no leaking internal service topology to the outside world.
Our Solution
We split the platform into three focused NestJS services behind an AWS Application Load Balancer, with the main server acting as an API gateway and orchestrator. It owns authentication and core domains, and delegates specialized work to the chatbot and health microservices over authenticated REST. A shared data and messaging layer keeps the services loosely coupled but consistent.
Architecture
- Main Server (NestJS) — API gateway and orchestrator: authentication, users, goals, recipes, scheduling, and notifications. All clients have a single public entry point.
- Chatbot Microservice (NestJS) — AI conversational service powered by Azure OpenAI (GPT-4o) via LangChain/LangGraph, with Elasticsearch-backed RAG for recipe and knowledge retrieval.
- Health Microservice (NestJS) — ingests and aggregates wearable and manual health data (Apple Health, Health Connect) and serves analytics.
- AWS ECS Fargate runs the three containerized services, each with its own CPU/memory sizing and scaling policy.
- Application Load Balancer terminates HTTPS and path-routes traffic to the right service.
- Shared data layer — MongoDB Atlas (primary store), Redis (cache/sessions), Elasticsearch (search), ActiveMQ (async notification delivery).
CI/CD — Docker multi-stage builds pushed to Amazon ECR, deployed to ECS as rolling updates.
Key Features
1. Orchestrator pattern. The main server is the only service exposed to clients. It authenticates every request, then makes internal service-to-service calls — so the backend topology stays private and the client integration stays simple.
2. Authenticated service-to-service calls. Inter-service communication is REST over a shared HTTP client, secured with per-service bearer API keys:
// Main server delegating an AI request to the chatbot microservice const reply = await this.microserviceClient.post( this.chatbotApiKey, // per-service bearer key `${this.chatbotUrl}/chat`, { user, question, sessionId, attachment }, ); |
3. Independent scaling per workload. Each service is a separate Fargate task definition with its own sizing — the memory-heavy chatbot service scales independently of the lightweight core API, so AI traffic spikes never starve everyday requests.
4. Right-sized AI service with built-in resilience. The chatbot service rotates across multiple Azure OpenAI keys, automatically failing over on rate limits or errors — keeping AI features responsive under load.
5. Notifications via asynchronous messaging. Because ActiveMQ delay queues separate time-based nudges and reminders from the request stream, notification delivery never hinders or slows down core API traffic.
6. Repeatable, isolated deployments. From ECR to ECS, each service is shipped as an own Docker image. A modification to the health service merely redeploys the health service—quick, low-risk releases with rolling updates that are health-checked.
7. One consistent client contract. Mobile (React Native) and the admin dashboard all talk to a single load-balanced, HTTPS entry point — the internal split into microservices is invisible to them.
Results
- These days, the workloads for AI chatbots, health data, and core APIs scale separately; no task can deteriorate the others.
- Each service deploys on its own, turning risky platform-wide releases into fast, isolated updates.
- Compute is right-sized per service, eliminating the over-provisioning of a one-size-fits-all server.
- A single secure, load-balanced entry point keeps client integration simple while the backend stays private and modular.
Technology Stack
NestJS · TypeScript · Node.js 20 · Azure OpenAI (GPT-4o) · LangChain · MongoDB Atlas · Redis · Elasticsearch · ActiveMQ · AWS ECS Fargate · Application Load Balancer · Amazon ECR · Docker
Other Blogs
1. How We Scale Video Processing Workloads with AWS ECS
2. How We Use AWS ECR to Manage and Deploy Container Images
3. How We Use AWS EC2 for Workloads in High-Performance Video
