One slider, three renderers: how we built Instagram-style video filters with GPU shaders for preview, Skia color matrices for everywhere else, and FFmpeg for export — and kept them all telling the same visual story.
Filters look easy. Video filters aren't.
Applying a sepia tone to an image in Flutter is a one-liner — wrap it in a ColorFiltered widget and you're done. Applying it to a video is a different sport entirely:
- The preview must run at 30–60 fps on top of a playing video, on whatever device the user owns.
- The user must be able to drag sliders (brightness, contrast, hue, temperature…) and see the result live.
- And the cruel part: when they hit Export, the burned-in MP4 must look like what they previewed — but your preview renderer (Flutter/GPU) and your export renderer (FFmpeg) are completely different engines that have never heard of each other.
This post is about how we built that pipeline in a Flutter video editor: a filter catalog with presets and per-parameter adjustments, a live preview that picks the cheapest renderer that can do the job, and an FFmpeg export path that reproduces the look. The headline lesson up front:
Every filter ends up implemented three times — as a GPU shader configuration, as a Skia 4×5 color matrix, and as an FFmpeg filter string — all driven by one shared parameter map. Maintaining the visual consistency of those three reimplementations is the true engineering.
The architecture: one parameter map, three renderers
┌────────────────────────────┐
│ shared parameter map │
│ {filterId, params, 0–1 │
│ intensity} │
└─────┬───────┬───────┬──────┘
│ │ │
┌──────────────┘ │ └───────────────┐
▼ ▼ ▼
GPU shader preview Skia ColorFilter.matrix FFmpeg -vf string
(flutter_gpu_video_ (primary live preview, (export — the only
filters, local clips) universal fallback) thing users keep)
The three paths exist because each one wins somewhere:
- GPU shaders (flutter_gpu_video_filters) give true shader-quality effects — distortions, blurs, tone mapping — but the package renders into its own surface and works best with local video files.
- Skia ColorFilter.matrix — a 4×5 color matrix wrapped around the video widget with ColorFiltered — is nearly free, works on any video widget (network streams included), and covers the most-used two dozen filters: brightness, contrast, sepia, grayscale, hue, duotones…
- FFmpeg is the only renderer whose output the user actually keeps. Export re-derives every look as a filter-graph string (eq=, hue=, colorchannelmixer=…).
A small strategy router decides per-filter which preview engine to use:
// Which preview strategy can render this filter?
static FilterStrategy getFilterStrategy(String filterId) {
if (_colorFilterIds.contains(filterId)) return FilterStrategy.colorFilter;
if (_customPainterFilterIds.contains(filterId)) return FilterStrategy.customPainter;
return FilterStrategy.exportOnly;
}
- colorFilter → wrap the player in ColorFiltered(colorFilter: getColorFilter(id, params))
- customPainter → stack a CustomPainter overlay on the video (vignette, pixelation, halftone — things a color matrix can't express)
- exportOnly → show the original video plus a badge telling the user the effect appears in the export
That router is the single most cost-effective decision in the system: it gives ~24 filters genuinely free real-time preview on every device, and reserves the heavyweight machinery for the filters that need it.
1. The filter catalog, presets, and adjustments
Each filter in the catalog is a tiny declarative item — an id, a display name, a category, and (when the GPU path supports it) a factory for the shader configuration:
const GpuVideoFilterItem({
required this.id,
required this.name,
required this.icon,
required this.category,
this.createConfiguration, // () => GPUFilterConfiguration, when GPU-capable
});
GPUFilterConfiguration? getConfiguration() => createConfiguration?.call();
Presets are just named parameter bundles pointing at a filter — the "P1–P5" quick looks in the UI:
static List<FilterPreset> get allPresets => [
const FilterPreset(id: 'P1', name: 'Warm Vintage',
filterId: 'sepia', parameters: {'intensity': 0.8}),
const FilterPreset(id: 'P2', name: 'Classic B&W',
filterId: 'grayscale', parameters: {'intensity': 1.0}),
const FilterPreset(id: 'P3', name: 'Vivid Colors',
filterId: 'vibrance', parameters: {'vibrance': 0.5}),
const FilterPreset(id: 'P4', name: 'High Contrast',
filterId: 'contrast', parameters: {'contrast': 1.4}),
const FilterPreset(id: 'P5', name: 'Cinematic',
filterId: 'vignette', parameters: {'vignetteStart': 0.3, 'vignetteEnd': 0.75}),
];
Adjustments are typed parameter descriptors with ranges, defaults, and even a gradient hint so each slider can render a meaningful track (black→white for brightness, a rainbow for hue):
case 'brightness':
return [const FilterParameter(id: 'brightness', displayName: 'Brightness',
minValue: -1.0, maxValue: 1.0, defaultValue: 0.0,
gradientType: FilterParameterGradientType.blackToWhite)];
case 'contrast':
return [const FilterParameter(id: 'contrast', displayName: 'Contrast',
minValue: 0.5, maxValue: 2.0, defaultValue: 1.0,
gradientType: FilterParameterGradientType.grayToWhite)];
case 'hue':
return [const FilterParameter(id: 'hue', displayName: 'Hue',
minValue: -180.0, maxValue: 180.0, defaultValue: 0.0, unit: '°',
gradientType: FilterParameterGradientType.rainbow)];
The current slider values live in an immutable state object, seeded from defaults:
factory FilterAdjustmentState.fromDefaults(String filterId, List<FilterParameter> parameters) {
return FilterAdjustmentState(
filterId: filterId,
parameterValues: Map.fromEntries(parameters.map((p) => MapEntry(p.id, p.defaultValue))),
);
}
And one UX detail that matters more than it looks: slider updates write to state immediately (so the thumb tracks the finger), but the preview re-render is debounced by 300 ms. Without the debounce, dragging a slider rebuilds the filtered video subtree dozens of times per second and the drag stutters; with it, the preview snaps to the final value the moment the finger slows down.
What gets persisted on the clip is deliberately minimal:
class Filters {
final String? adjust; // manually chosen filter id, e.g. 'sepia'
final String? presets; // preset id, e.g. 'P1'
final double? intensity; // 0.0–1.0; null means 1.0 (only saved when < 1.0)
final Map<String, double>? parameters; // per-filter slider values
}
2. Live preview, path by path
The workhorse: a 4×5 color matrix
Most of the catalog is color math, and Skia color matrices do color math for free. The filter manager turns a filter id + parameters into a ColorFilter.matrix:
static ColorFilter getColorFilter(String filterId, [Map<String, double> parameters = const {}]) {
final matrix = _getColorMatrixWithParams(filterId, parameters);
return ColorFilter.matrix(matrix);
}
The matrices of parameterized filters are constructed dynamically.Two examples — and note the ×255, because matrix offsets live in 0–255 color space:
case 'brightness':
final brightness = (params['brightness'] ?? 0.0) * 255.0;
return [1,0,0,0,brightness, 0,1,0,0,brightness,
0,0,1,0,brightness, 0,0,0,1,0];
case 'contrast':
final contrast = params['contrast'] ?? 1.0;
final offset = -(0.5 * contrast) + 0.5;
return [contrast,0,0,0,offset*255, 0,contrast,0,0,offset*255,
0,0,contrast,0,offset*255, 0,0,0,1,0];
Filter intensity — the global "how much of this look" slider — is implemented as a straight interpolation between the effect matrix and the identity matrix:
final interpolatedMatrix = List<double>.generate(20, (i) {
return identityMatrix[i] + (effectMatrix[i] - identityMatrix[i]) * intensity;
});
That one-liner gives every color filter a free 0–100% strength control with zero extra shader or pipeline work.
The real GPU path
For local clips, the editor swaps in a true GPU-filtered surface from flutter_gpu_video_filters. Because the widget is keyed, altering the clip or filter causes the entire surface to be destroyed and recreated (the package's controller dislikes hot configuration switches mid-flight):
return GPUVideoSurfacePreview(
key: ValueKey(filterKey), // '${clipIndex}_${filterId}' — force full recreation
configuration: _gpuFilterConfiguration!,
onViewCreated: (controller, sizeStream) async {
_gpuPreviewController = controller;
if (_mainController != null && _isPlaying) {
_mainController!.pause(); // avoid duplicate playback (and doubled audio!)
}
await controller.setVideoSource(FileInputSource(File(clipInfo.assetPath!)));
if (mounted) setState(() => _isGpuPreviewInitialized = true);
},
);
The shader configuration is assembled per filter from the package's typed configs, fed by the same parameter map the sliders write into:
case 'brightness': return GPUBrightnessConfiguration()..brightness = getParam('brightness', 0.0);
case 'contrast': return GPUContrastConfiguration()..contrast = getParam('contrast', 1.0);
case 'saturation': return GPUSaturationConfiguration()..saturation = getParam('saturation', 1.0);
case 'hue': return GPUHueConfiguration()..hue = getParam('hue', 0.0);
case 'white_balance': return GPUWhiteBalanceConfiguration()..temperature = getParam('temperature', 5000.0);
Two lifecycle lessons paid for in debugging hours:
- Pause the underlying player before the GPU surface starts. The GPU preview plays the video itself; forget the pause and you get two decoders playing the same clip — including doubled, slightly offset audio.
- Dispose properly on filter change: disconnect() the old configuration, create new preview params, connect() the new one, and dispose() the controller (plus cancel the size-stream subscription) when leaving the screen.
The desktop curveball
On desktop, the video backend renders into an external texture that bypasses Skia's layer compositing — so wrapping the player in ColorFiltered silently does nothing; there's no Skia layer to transform. The workaround is to force rasterization of the underlying layer (e.g. an interposed BackdropFilter over a SizedBox.expand) so the color matrix has actual pixels to operate on. If your filters "work on Android but not on desktop," this is almost certainly why.
3. Export: rebuilding the look in FFmpeg
The preview is rented; the export is owned. At export time, the saved Filters model is resolved (a manually chosen filter takes precedence over a preset; preset ids are mapped back to their filter + parameters) and translated into an FFmpeg filter string:
switch (normalizedId) {
case 'grayscale': return 'hue=s=0';
case 'sepia': return 'colorchannelmixer=.393:.769:.189:0:.349:.686:.168:0:.272:.534:.131';
case 'invert': return 'negate';
case 'brightness':
final b = params?['brightness'] ?? 0.3; return 'eq=brightness=${b.toStringAsFixed(3)}';
case 'contrast':
final c = params?['contrast'] ?? 1.5; return 'eq=contrast=${c.toStringAsFixed(3)}';
case 'saturation':
final s = params?['saturation'] ?? 1.5; return 'eq=saturation=${s.toStringAsFixed(3)}';
case 'exposure':
final e = params?['exposure'] ?? 0.4; // FFmpeg eq has no exposure — fake it with gamma
final gamma = e >= 0 ? (1.0 - e * 0.5).clamp(0.1, 10.0)
: (1.0 / (1.0 + (-e) * 0.5)).clamp(0.1, 10.0);
return 'eq=gamma=${gamma.toStringAsFixed(3)}';
case 'hue':
final h = params?['hue'] ?? 90.0; return 'hue=h=${h.toStringAsFixed(1)}';
case 'gaussian_blur':
final sigma = (params?['sigma'] ?? 5.0).clamp(0.1, 50.0);
return 'gblur=sigma=${sigma.toStringAsFixed(1)}';
case 'vignette': return "vignette='PI/4'";
// swirl / bulge / toon / kuwahara / crosshatch ... → return '' (preview-only)
}
Intensity at export: the split/blend trick
Remember the preview implements intensity by lerping the color matrix toward identity. FFmpeg has no "matrix lerp" — but it has stream compositing. So a partial-intensity export splits the video, filters one branch, and blends it back over the original at the saved opacity:
if (intensity >= 0.99) {
command = ['-i','"$videoPath"','-vf', filterCommand,
'-c:v','libx264','-preset',options.preset,'-crf','${options.crf}',
'-c:a','copy','-movflags','+faststart','-y','"$outputPath"'].join(' ');
} else {
final opacity = intensity.toStringAsFixed(2);
command = ['-i','"$videoPath"','-filter_complex',
'[0:v]split[orig][tofilter];'
'[tofilter]$filterCommand[filtered];'
'[orig][filtered]blend=all_mode=normal:all_opacity=$opacity[out]',
'-map','[out]','-c:v','libx264','-preset',options.preset,'-crf','${options.crf}',
'-c:a','copy','-movflags','+faststart','-y','"$outputPath"'].join(' ');
}
It's not mathematically identical to the matrix lerp, but perceptually it lands close — and it works for any filter string, not just color matrices.
The filter pass is one isolated re-encode in a longer export chain (canvas-fit → filter → effects → overlays → audio mix → concat), with -c:a copy keeping audio untouched until the dedicated mixing stage, and the source duration probed at every stage to catch drift early.
4. The parity table: where three renderers agree — and where they don't
This is the part nobody writes about. One stored value, three interpretations:
| Parameter | Slider range | GPU shader | Skia matrix | FFmpeg export |
|---|---|---|---|---|
| brightness | −1 … 1 | native −1…1 | offset = v × 255 | eq=brightness=v |
| contrast | 0.5 … 2 | native | diagonal v, offset re-centered | eq=contrast=v |
| saturation | 0 … 2 | native | Rec.709 luma-weighted blend | eq=saturation=v |
| exposure | −1 … 1 | linear gain | linear gain | faked via gamma curve |
| hue | −180° … 180° | native | full cos/sin rotation matrix | hue=h=v |
| sepia | intensity | shader | matrix .393/.769/.189… | colorchannelmixer — same coefficients, exact parity |
| intensity | 0 … 1 | preset multiplier | lerp matrix → identity | split + blend=all_opacity |
Hard-won parity lessons:
- Pick one filter as your ground truth. Sepia achieves exact parity because the identical 3×3 coefficients appear in both the Skia matrix and FFmpeg's colorchannelmixer. Build that one first and use it to validate your pipeline end to end.
- Beware unit mismatches. The Skia brightness offset is in 0–255 space while the GPU shader works in −1…1 — the same slider value is dramatically more aggressive on one path unless you normalize deliberately.
- Some mappings are approximations, and that's a decision. FFmpeg's eq has no exposure control, so export approximates exposure with an inverse gamma curve. Linear-gain preview vs. gamma-curve export diverge in the shadows. Acceptable? Maybe — but decide it consciously and write it down.
- Watch for parameters that silently degrade. In an early version, the white-balance Kelvin slider drove the GPU preview beautifully — while the export emitted a hard-coded color temperature. The user dragged a slider that did nothing to their final video. Audit every parameter across all three paths.
- Preview-only filters must say so. Distortion and stylize effects (swirl, bulge, toon, kuwahara…) exist only as GPU shaders; their export translation returns an empty string and the pipeline passes the original video through. The UI has to disclose this honestly — a silent mismatch between preview and export is the fastest way to lose a user's trust.
5. Performance notes from production
- Don't GPU-filter what a matrix can render. The strategy router sends ~24 color filters through ColorFiltered — effectively free, works on network streams, works everywhere Skia composites. The GPU surface is reserved for local files and shader-only effects.
- Key the GPU surface; don't mutate it. ValueKey('${clipIndex}_${filterId}') and full recreation beats trying to hot-swap shader configurations on a live controller.
- Debounce sliders (≈300 ms) — update state immediately for a responsive thumb, re-render the filtered preview lazily.
- Skip per-filter thumbnail rendering unless you need it. Our filter carousel uses icon tiles instead of 30 filtered video thumbnails; generating and caching real filtered previews per filter per clip is a deceptively large cost on low-end devices for marginal UX gain.
- Persist sparsely. Intensity is only written when < 1.0; null means "full strength." Tiny choices like this keep per-clip JSON lean when projects have dozens of clips.
Lessons learned
- There is no single "filter implementation." Accept that preview and export are different engines; design one canonical parameter map and treat every renderer as a projection of it.
- Route by capability. A three-tier strategy (color matrix → custom painter → GPU/export-only) gives most filters free real-time preview and saves the expensive path for effects that earn it.
- Intensity is the cheapest premium feature you'll ever ship — a matrix lerp in preview, a split/blend in FFmpeg.
- Parity is a test surface. Render a frame through each path with the same parameters and compare. The mismatches you'll find (units, gamma vs. gain, hard-coded constants) are exactly the ones users would have found for you.
- Be honest in the UI. If a filter only exists at export — or only in preview — label it. Trust survives missing features; it doesn't survive surprises.
The result: a filter system with live preview on everything from a budget Android phone to a desktop build, slider-level adjustments with instant feedback, named presets, and exports that look like what the user saw — built from one parameter map and three carefully reconciled renderers.
