Realistic AI video: let the language model do the physics

A pretty picture and correct physics are two different skills, and diffusion video models are much stronger at the first. In K2V, a paper from Cambridge and Toyota, the physics moves to a language model: it reasons in words, sets keyframes at the moments that matter, and leaves the gaps to the video model. Here is how it works, what a clip costs, and which part of it you can use today.

A strip of frames where three ball positions are fixed and the empty gaps are left to the video model
01

What K2V actually does

On 12 November 2026 a paper called «K2V: Generative Keyframing For Physically Plausible Video Generation» landed on arXiv. The authors are Felix O'Mahony, Daniel Olmeda Reino, Roberto Cipolla and Ayush Tewari, from Cambridge University and Toyota Motor Europe, paper number 2611.17324. The title reads like a chore. Behind it sits a redistribution of work between two models.

One video model usually does everything: reads the prompt, invents the motion, draws the frames. K2V splits the job. A language model reasons in words about what has to happen, then builds keyframes at the points where physics is decided: an impact, a fall, a touch. The video model takes those frames as a condition and fills the gaps between states that are already correct.

The framing is honest. Nobody claims the video model will learn to calculate: the calculating goes to whatever reasons in text, the drawing goes to whatever draws.

02

Why diffusion models get physics wrong

A video model is trained to look plausible, it does not compute motion. A lovely arc for a flying ball is as acceptable to it as the real parabola. The error is not always obvious: the shot feels alive while a hand passes through a table or water pours into a glass at an odd angle.

Physics-IQ, VideoPhy-2, PhyGenBench and PhyWorldBench exist to measure exactly that, and the authors lean on them. Our own look at the physics benchmark behind PhyRubric shows even strong models scoring far from perfect. These tests give a percentage of hits instead of the phrase "looks convincing".

Three stations: reasoning in words, keyframes with one anchor, then the video model filling the clip
03

The split: words, frames, gaps

Three roles live inside K2V, and that is the interesting part.

The planner. An agent on Gemini 3.8 Flash makes several calls in a row, writes out what should happen along the motion, and decides which states matter. Table 4 in the paper reports 6.1 calls per Physics-IQ task and 10.1 per PhyGenBench task.

The keyframe artist. Gemini 3.1 Flash Image edits and repaints frames at those anchor points: 1.7 images on Physics-IQ and 2.9 on PhyGenBench.

The renderer. Wan2.2-14B takes the finished frames as a condition and generates the clip once, filling the transitions between them.

Keyframes in AI video normally sit at the start and the end of a shot. Here they are scattered through the action, right where the model tends to fail. The request becomes «hold these three states» in place of «make it pretty».

04

What the numbers say

The paper reports clearly better results than existing approaches, with the best score on Physics-IQ Verified and PhyGenBench. Those are the authors' own runs on two tests, with no peer review and no outside check, and that caveat is worth keeping in mind.

The weak spot is predictable. Once the reasoning belongs to a language model, quality depends on how precisely it described the physics. Misread the collision and the keyframe comes out just as wrong, only rendered with confidence.

One clip with two coins on the left, four clips with a taller coin stack on the right: one pass is cheaper
05

What a clip costs

The authors did the arithmetic, a rare courtesy in research papers. Table 4, prices as of 8 October 2026.

Video generation: one Wan2.2-14B clip for $0.165. Planning and frame edits are billed per call and per image, which brings a Physics-IQ clip to $0.47 and a PhyGenBench clip to $0.59.

The paper also prices a system called NEWTON at $2.93 per clip, since it runs four passes of Seedance 2.0 Fast. That is five to six times the cost of K2V: here the video model runs once instead of four times, and it is an open model.

I think this matters more than the benchmark. While a clip costs cents, you can afford attempts and retakes, and the quality shows up in the retakes.

06

Where it falls short

K2V is research. There is no product: no service, no subscription, no job queue, no promise the approach ships in this shape. Rebuilding it yourself only goes part of the way: the language model and the image generator are available, while the link to an open video model has to be assembled on your own machine.

Second, the table compares money against NEWTON while quality is measured on the authors' own tests. That says little about your footage, where the frame holds a face, a package and the light.

Third, the physical plausibility metric is still coarse. As long as it is a percentage over a set of scenes, winning it means winning on that set.

07

Keyframes by hand: what you can copy now

The useful takeaway is about process, and it works without any K2V.

Break the motion into moments where something is decided. An impact, a fall, a touch, a change of direction. Those are the three or four points where a model usually lies.

Build a frame for each point: draw it, pull it from a reference or compose it in an editor. The pose and the contact have to be right, since the model will lean on that frame.

Leave the rest to the video model: transitions, blur, camera movement. It handles those more reliably once the anchor points are set.

Check the seams. Look at the way into a keyframe and the way out of it, and you will see whether the model holds the physics or starts inventing.

That is also the argument for building important shots by hand. When a clip is made with AI video production, the keyframes in the expensive places are drawn by a person, and the bad takes do not reach the feed. For advertising this matters most: an AI commercial with an impact, a pour or a fall almost always has one moment where physics is visible up close. Our handcrafted videos start at $190, and a person owns those frames.

08

My conclusion

The split of labour looks more important to me than the exact percentages. For years one video model was expected to do two different things: calculate and draw. K2V shows they can be separated, and it does so with a cheap chain of a language model, an image generator and an open video model.

The practical takeaway for this month is simple. Keyframes at the moments that matter pin down physics more reliably than a long prompt describing it. If a scene has an impact, a fall or a touch, set a frame on that moment instead of hoping the model guesses.

The order of work, anchor states first and transitions after, is worth taking now, because it does not depend on where the K2V story ends.

frequently asked

01What is K2V
A research paper from Cambridge University and Toyota Motor Europe, posted on arXiv on 12 November 2026 as 2611.17324. The authors set keyframes with a language model and leave only the transitions to the video model.
02How is this different from ordinary keyframes
Usually keyframes define the first and the last frame of a shot. Here they are placed through the action, at the moments where physics is decided: a collision, a fall, a touch. The video model then interpolates between states that were already checked.
03Can I use K2V today
No. It is a paper with no public access: no service, no price, no dates. The keyframe habit can be copied by hand, and the full chain has to be assembled around an open video model.
04Why do video models fail at physics
They are trained on how frames look, with no motion calculation in the loop. To such a model a beautiful trajectory equals a correct one until a test checks the result. That is why Physics-IQ, VideoPhy-2, PhyGenBench and PhyWorldBench exist.