Driving GTA V with a neural net: a world model on one GPU

This card interested me more than another text-to-video announcement. lucidml published a network that keeps GTA V going while you hold WASD: 630,205,008 parameters, Apache-2.0, one consumer GPU. Here is how it remembers past frames, why a session stops after three minutes, and why it is not a tool for client work yet.

A chain of game frames: start image on the left, action arrows, world continued frame by frame
01

What lucidml actually published

On 11 November 2026 a small independent lab called lucidml showed a network playing GTA V on X, and on 12 November the card lucidml/gta appeared on Hugging Face. The repository holds 630,205,008 parameters and the code to run them under Apache-2.0. There is no training code in it. There is no browser client in the repository either, the demo lives separately. The author writes the card in the first person, and the tone reads like a lab diary instead of a press release.

Training ran from scratch on a dataset called GG800, which is 800 hours of recorded GTA V.

02

Making video in games: what a world model means

A normal video model takes a text prompt and draws a clip a few seconds long. A world model works differently. It predicts the next frame from the current state and from the player action. The action here is a key press, five numbers. So the same spot in the game can be reached along two different routes, and the network draws out both.

In plain words, it is a learned simulator. The network watched 800 hours of the GTA V world behaving, and now it continues that world using the rules baked into its weights.

That is where the interest comes from: making video in games used to mean a screen recording or a capture edit. Here the video stream appears the moment a key goes down. We covered the same idea in a different field in our piece on world models for robots, where a network predicts what a robot arm will do next.

Memory diagram: recent frames full, older frames compressed into summaries, start image as anchor
03

What it feels like to drive

Startup is simple. You hand it one starting image, then hold WASD, and the model generates the next video frame by frame as it streams into a browser. There is no text prompt at all. The card lists the conditioning: the diffusion step, the player action, the frame rate and the grid size. Each frame costs three denoising passes plus one clean pass that is written into memory.

The output is modest: 384×672, an H.264 stream over WebSocket. It needs CUDA and torch with FlexAttention, and one GPU process serves one player. The author says it runs on a single consumer graphics card.

04

How the model remembers past frames

The architecture is called Summaryformer. Each step is a whole frame. At every step the model looks at the current frame, at the two most recent frames in full resolution, at older frames inside a 24 frame window, and at the starting image, which is always kept.

The older frames enter the calculation in compressed form. They become 96 summary tokens, and 6,528 keys do the job that would otherwise take 26,208. That cuts the amount of computation at every step. The network stacks 28 blocks at a width of 1280.

The memory window is 24 latent frames, which equals 96 video frames. Anything earlier survives only in the summaries and in the starting image. The starting image acts as an anchor: the scene keeps its overall look while memory washes the detail out.

A strip of frames slowly warping, scissors cutting the session, a clock face without numerals
05

Why a long session drifts

A session is cut at 180 seconds by default, with a ceiling of 4,096 frames. That is an honest admission of the limit. Short memory holds recent motion, and past that point the picture collects error and drifts, which the write-up of the card warns about directly.

A similar logic helps explain why faces and hands drift in long AI videos. The model leans on a handful of recent frames while the rest of the context is compressed. We wrote about that separately, and the conclusion matches: a long shot is safer when it is built from short pieces that share an anchor at the start.

06

Why this is still a toy

Three reasons, all of them on the card. First, training used recordings of GTA V, the author does not discuss the rights to that footage, and such a base does not work for a commercial product. Second, there is no training code, so the model cannot be fine tuned on your own game, and all you can do is run what shipped. Third, 384×672 output and short memory mean the picture drifts on long sessions.

For someone ordering video, that adds up to one thing: there is no tool here you can bill for. There is a demonstration of an idea.

07

What a video order can take from it

The idea is useful, and it already runs inside ordinary generation. A strong opening frame as the anchor, short memory of recent frames, a summary of older context. For AI video production that suggests a clear order: a solid anchor frame, short pieces, a check on every seam. For a series on an AI YouTube channel it means a long episode gets cut into pieces that share one anchor instead of being generated in a single pass.

The money side is known: handcrafted videos from $190, short form video at scale up to $2.40 per 1000 views, and a managed AI YouTube channel from $200 per month.

08

My take

I would keep this card bookmarked as a teaching example. It is not a working tool. The value sits in the explanation: you can finally see what interactivity is made of and why a 96 frame memory caps everything else.

If a model like this lands with open training code and hours of your own footage, the conversation changes. For now 630 million parameters, one player per GPU and Apache-2.0 are good news for researchers and neutral for production.

frequently asked

01Can I shoot a client video on this model
No. The card does not settle the rights to the GTA V footage, there is no training code, and fine tuning on your own game is not available.
02What is a world model in plain words
A network that predicts the next frame from the current state and the action the player takes. The rules of the world live in its weights, so one place can be reached along different routes.
03Why do long sessions fall apart
Memory holds a 96 frame window plus summaries of older context. Error accumulates and the picture drifts. The default session is cut at 180 seconds, a server setting that can be switched off.
04Do I need a world model to make a video about a game
A screen recording or a keyframe driven generation is enough for a clip. A world model matters where the viewer controls the picture.