Face swap in video: two jobs under one name
Two different technologies answer to the phrase face swap, and confusing them costs money. The first works frame by frame: a model finds the face, transfers features from a donor photo, blends the result into the original frame and pushes the frame back into the stream. Inside that pipeline you always find face detection, tracking across frames, a skin mask and colour matching against the light of the scene.
The second arrived with generative models and goes by video-to-video. The model takes the source clip and a set of references, then redraws the whole scene. The character changes along with clothing, light and background, because to the model this is one picture in motion.
The practical difference matters. Classic face swap is cheaper and holds one specific face with more accuracy. Video-to-video keeps the scene better and rewrites things you never asked it to touch.
What Kling 4.0 and Pika H3 Flex changed
Two releases from the past week explain why the topic is back at the top of every feed.
On October 23, 2026 the Runway Dev documentation picked up an entry for Kling 4.0: the model edits source video, accepts up to 10 frames and references, plus up to 5 reference videos. The model now corrects finished footage against a sample, and a face replacement becomes one case of that correction.
On October 29, 2026 Pika shipped a press release for H3 Flex. The company's wording: precise editing, replacement of characters, camera moves, look and voice. That claim reaches past a face swap, and it shows the direction: rewrite an existing clip from a reference while keeping the motion.
For a client the conclusion is short. A year ago a face swap was a separate job with separate software and a separate invoice. Today it is a feature inside a generative model and costs like ordinary generated seconds.
Why the face still flickers
The technical snags have barely moved in two years, and they eat more hours than the swap itself.
Flicker shows up on head turns: the tracker loses its anchor points, the mask jumps, skin tone drifts between frames. Profile shots and half turns stay the weak spot of nearly every model, since the donor photo is usually shot straight on. A hand or a microphone across the face tears the mask and leaves a ragged edge. Hard side light pulls highlights, and the transplanted face starts to shine where the scene does not.
Hair and glasses deserve their own paragraph. Thin strands against a bright background get redrawn the model's way, and in motion it reads as crawling. Spectacle frames double up when the reference has none.
Lips are simple until sync enters. A face replacement keeps the original articulation, so dubbing into another language drifts away from the picture. That needs a separate lip sync pass, which is different work with a different budget.
Where consent comes in
This is the part that cancels projects.
In most jurisdictions the person in the frame holds the right to their own likeness. In the United States that right travels under the name publicity and in some states it outlives the person. In the European Union a face counts as personal data under the GDPR, so a video of someone is processing their data and needs a lawful basis. Consent is the cheapest basis you will ever find, and the only one that survives a change of vendor.
Several national codes spell the same thing out: publication and use of a person's image, video included, need consent. The exceptions are narrow: state or public interest, filming in places open to the public where the person is not the main subject, and posing for payment.
Without consent the person can demand the footage come down, and damages follow. One honest caveat: enforcement around deepfakes is still forming, and a dispute will more often turn on whether the person is recognisable in the frame than on the fact of generation. Nobody can promise you the outcome. This is a reading of the rules, and it is not legal advice.
Reputation is the second door
A face swap that damages someone's honour and reputation opens a second front. Civil codes in most countries give the person a claim for refutation and removal, and where the facts fit, criminal defamation or fraud enters the picture. That case no longer argues about consent at all.
The practical lesson: consent closes the first door, reputation closes the second, and the second opens without your help. A clip with someone else's face in an unflattering context outlives its removal, because screenshots do not come down.
Four setups that hold up
Your own face. An in-house expert talks with their own face across a dozen videos, and no consent is needed because the person is both the client and the subject.
An actor under contract. The agreement carries a separate clause on the digital double: where it may be used, for how long, on which platforms. That clause matters more than the fee, since the actor leaves in a year and the footage stays.
Localisation. A host is replaced with another host under consent from both sides: the one removed and the one added sign paperwork. The setup works well for entering markets that want a local speaker.
An AI character with no real prototype. No face, no consent, and the calmest route for serial formats. Building such a character and keeping it recognisable across scenes is covered in our piece on character consistency.
Labeling and trust
Then there is AI content labeling. Platform rules on disclosure keep shifting, and on a face swap clip it is the first question an ad review asks, because the audience wants to know who is on screen. What is mandatory right now and how it looks in practice is laid out in our guide to AI content labeling.
My take: the swap is easy, consent is the work
Swapping a face is technically easy now. A model takes the source and a reference, returns a result in minutes, and bills like any generated footage. The hard part sits elsewhere: getting consent, writing it so that it survives a change of contractor, and pushing the shot past the flicker stage.
The budget picture is plain. One video a month runs fine on a subscription. A social series with ad variants needs another setup: one template, many versions, a predictable cost per unit, which is where AI video production on a fixed pipeline wins. Where a real person has to be on camera and their face carries the trust, handcrafted videos from 200 dollars are the safer line: the price is known up front and does not move with the number of attempts.
When the face has to be your own and the volume is high, both live in one project: the expert talks to camera, while the wrappers, cutaways and platform versions come out of generation. Start with one short test on your own footage, five or six shots with different head angles, and you will see exactly where the model breaks. That costs less than discovering the flicker on a finished series.
