The Control-Net Method: A Hybrid AI Video Production Workflow That Kept Creative Control

When 40% of the Budget Vanished

A hybrid AI video production workflow is not a thought experiment when the money runs out mid-shoot. The Singapore drama series Crooks was already in production when one of its investors withdrew around Chinese New Year, taking roughly 40% of the funding with it. Grant deadlines were fixed and the team had few options, so producer Boi Kwong and his collaborators spent three months building an AI-assisted pipeline from scratch.

For Kwong, whose credits include Geylang (2022) and Sui Yue: The Days (2008), the point was never to let the machine make the film. 'It is a very curated and conscious way of using AI to control our output,' he told Digital Edge. The constraint became the catalyst: a smaller budget forced a workflow where AI absorbed the expensive, repetitive work and humans kept the decisions that actually define the film.

The Consistency Problem No Single Prompt Solves

The first hard limit the team hit is the one every generative pipeline meets eventually. Large image and video models can produce a convincing single shot, but they begin to break down as scenes get crowded and the camera moves. AI researcher Lim Kim Hui puts it bluntly: give the model too much information in one generation and 'it will start to move and hallucinate.' Instead of only swapping a background, it quietly rewrites camera angles, characters and composition.

That drift is why a generated face or a product can look right in frame one and wrong by frame four. A reference-first {{link}} is what keeps a generated face from drifting between cuts. Crooks treated consistency as a structural problem to design around, not a flaw to regenerate away one clip at a time.

A reference-first AI video character consistency is what keeps a generated face from drifting between cuts.

Break the Shot Into Layers, Then Recombine

Rather than asking one prompt to deliver a finished scene, the team borrowed a technique from traditional visual effects: they split each shot into separate layers. Backgrounds, lighting and performances were generated and processed individually, then recombined. VFX supervisor Jay Hong would repaint lighting by hand in After Effects and Nuke, then push the shot back through the models for a physically correct final render.

This is the unglamorous core of the method. The recombine step is where {{link}} stops being optional. The model is only ever trusted with one layer at a time, which is why keeping generations narrow made them controllable: a complex shot that once needed thirty or forty attempts settled into about six, and simple close-ups often landed in one or two.

The recombine step is where editing AI-generated video stops being optional.

The Control Net: A Hybrid AI Video Production Workflow That Stays on a Leash

To stop the model from quietly rewriting the director's choices, the team built what they call a 'control net' - a framework of guides and seed values that tells the AI exactly what it can alter and what must stay untouched. 'A guider, or a seed number, that tells the AI what to change and what not to change,' Lim explains. The control net is the leash; the layers are the rope.

A written set of guardrails is the difference between a control net and chaos, which is the core of any {{link}}. Crooks documented its guides up front, which is also what makes the output auditable later. When a regulator, a broadcaster or a client asks how a frame was made, a team with a control net can answer; a team that winged it cannot.

A written set of guardrails is the difference between a control net and chaos, which is the core of any AI video governance playbook.

What the Numbers Actually Look Like

The savings were concrete, not theoretical. Lim says the team cut the number of generations needed for a usable shot from as many as thirty or forty in the early stages to about six, and that work which 'usually took 10 hours now takes one.' Kwong estimates lighting work alone dropped by between 50% and 70%, and overall savings across sets, logistics, post-production and equipment landed near 30%.

Crooks is one data point in a wider shift tracked by anyone modelling {{link}}. The pressure behind it is industry-wide: IAB's 2026 Digital Video Ad Spend report projects US digital video ad spend to surpass $80 billion this year, growing 11% year over year and accounting for more than 60% of total TV/video ad spend for the first time, with two in three buyers already live, testing or planning agentic AI for video. When buyers automate the top of the funnel, production teams feel the squeeze to ship more for less - which is exactly the condition that rewards a controlled, hybrid pipeline.

Crooks is one data point in a wider shift tracked by anyone modelling AI video production cost in 2026.

The Legal and Rights Questions Nobody Has Solved

The workflow is not a free pass on rights. Lim notes that 'all the actors are training the AI models' and that leading AI companies still face lawsuits and cease-and-desist demands from Hollywood studios over training data and creative works. Kwong's own view is pragmatic: audiences care far less about how a film is made than whether it works on screen, but that does not settle who owns the synthetic extensions of a performer's image.

The unsettled training-data lawsuits are exactly why teams document clearances in an {{link}} review. Doing it before they scale a control-net pipeline keeps the synthetic extensions of any performer's image defensible. Provenance matters here too: standards like C2PA's Content Credentials attach a tamper-evident record of a file's origin and edit history, so a generated environment or a cloned extra can be traced and disclosed instead of denied.

The unsettled training-data lawsuits are exactly why teams document clearances in an AI video commercial rights review.

What Commercial Teams Can Steal From Crooks

Strip away the drama-series context and the playbook transfers straight to commercial video. Fund the human producer, not the prompt: one person should own the brief, the brand and the guardrails end to end, with AI accelerating each stage. Decompose before you generate - backgrounds, lighting and talent as separate layers - so the model is never handed the whole frame and told to be creative.

Lock a control net with seed values and explicit 'do not touch' guides for anything brand-critical, and keep a written governance pack so the output is auditable. Measure the win in generations per usable shot and hours per task, not in a vague sense of speed. Crooks did not go full AI and survive; it went hybrid, stayed on a leash, and shipped. That is the part worth copying.