Serverless Video Processing Pipeline with AWS MediaConvert
The video platform needed a scalable, cost-effective way to handle variable encoding workloads, from quiet periods with few uploads to peak times with hundreds of simultaneous jobs.
Discuss Your Project
The Challenge
Fixed-capacity encoding infrastructure was either over-provisioned (costly) or under-provisioned (slow):
- Encoding workloads were highly variable and unpredictable
- Peak periods could see 100x normal volume during content launches
- Running dedicated encoding servers 24/7 was expensive during quiet periods
- Job failures needed automatic detection and retry without manual intervention
Our Solution
We implemented a serverless encoding pipeline using AWS Lambda triggers and AWS MediaConvert for elastic, pay-per-use video processing.
Architecture
- Trigger: AWS Lambda function monitoring S3 upload events
- Encoding: AWS MediaConvert with partner-specific job templates
- Messaging: ActiveMQ/STOMP for asynchronous job status updates
- Monitoring: NestJS encoder backend tracking job progress
- Storage: AWS S3 for input/output assets
Pipeline Flow
- S3 Event - Video upload triggers Lambda function
- Job Configuration - Lambda reads partner profile and constructs MediaConvert job
- Submission - MediaConvert job submitted with appropriate output settings
- Progress Tracking - STOMP messages relay status to encoder backend
- Completion - Output assets stored in S3, metadata updated in MongoDB
- Error Handling - Failed jobs queued for retry with exponential backoff
Key Features
- Zero Idle Cost - Lambda and MediaConvert charge only for actual usage
- Elastic Scaling - Handles 1 to 1000+ concurrent encoding jobs
- Partner Templates - Pre-configured MediaConvert job templates per partner
- Event-Driven - S3 events automatically trigger encoding workflows
- Comprehensive Monitoring - Job status, duration, and error tracking
Results
Technology Stack
caseStudyDetail.more Case Studies
Explore more of our technical implementations
Client-Side Ad Insertion (CSAI) with SCTE-35 Marker Parsing & Multi-Platform Player Integration
A video streaming platform needed to implement Client-Side Ad Insertion (CSAI) across web, mobile, and connected TV apps — enabling personalized, device-level ad experiences with full ad interaction support (clickable overlays, companion banners, skip buttons) that server-side insertion cannot provide.
SCTE-35 Ad Marker Signaling & Media Trailer Insertion Pipeline
A streaming media company needed a robust, automated pipeline for injecting SCTE-35 ad markers into live and VOD streams, along with the ability to insert promotional trailers (pre-roll, mid-roll, and post-roll) at precisely timed positions — enabling monetization across FAST channels, live events, and on-demand content libraries.
AWS Media Services for FAST Channel Streaming over SRT
A media company needed to establish reliable, low-latency contribution feeds for their FAST channels using the Secure Reliable Transport (SRT) protocol — enabling high-quality content ingest from remote studios, cloud playout systems, and syndication partners over unpredictable internet connections.
Frequently Asked Questions
MicrocosmWorks designed a segmented processing architecture where Step Functions orchestrate the pipeline: Lambda functions split source videos into segments, AWS MediaConvert handles the actual transcoding without Lambda timeout constraints, and a final Lambda stitches the output. This hybrid approach keeps the serverless cost model while supporting videos of any duration.
MicrocosmWorks measured a 70-85% cost reduction for bursty video processing workloads compared to running dedicated EC2 encoding instances. The serverless pipeline incurs zero cost when idle and scales to hundreds of concurrent jobs during peak periods, with AWS MediaConvert's per-minute pricing eliminating the need to provision for peak capacity.
MicrocosmWorks configured AWS Step Functions with per-step retry policies and exponential backoff, ensuring that a failed transcode step retries automatically without restarting the entire pipeline. Each stage writes intermediate outputs to S3, so recovery resumes from the last successful checkpoint rather than reprocessing from the source file.
MicrocosmWorks optimized the pipeline for near-real-time use cases with cold start mitigation using provisioned concurrency on critical Lambda functions and MediaConvert reserved transcoding slots. For live workflows, the pipeline achieves 2-5 minute end-to-end latency from upload to delivery, which is suitable for clip extraction and highlights distribution.
MicrocosmWorks builds serverless video infrastructure at rates of $25-$45/hr, with a complete MediaConvert-based pipeline including Step Functions orchestration, S3 lifecycle management, and monitoring typically requiring 250-400 development hours. The architecture's pay-per-use model means clients only pay AWS costs proportional to their actual processing volume.
Ready to Transform Your Business?
Let's discuss how we can apply similar solutions to your challenges.