How AI video background removal works

How does an AI know which subject you mean?

It starts from your clicks: a segmentation model works out the area of the image that belongs to the clicked object. That's called prompted segmentation.

At Keyotte, that model is SAM 2.1, released by Meta under an open license. Remove mode gives the model a point to exclude, to take out an extra area. Two or three well-placed clicks usually do it: one in the middle of the subject, one on a part that might be missed (a hand, a held object).

How is the subject tracked from frame to frame?

The model keeps a memory of what the subject looks like and finds it again on the next frames, even as it moves or changes pose. That's tracking.

When you fix a frame with a click, that information joins the memory and is used from then on. That's why one fix in the right place often repairs a whole stretch.

How are the edges refined?

The tracking mask is a bit rough around the outline. A second network, specialized in fine cutouts (matting), recomputes the edges frame by frame, hair included; at Keyotte, that's BiRefNet.

Temporal smoothing then stops the edges flickering from frame to frame. This step takes the most computing: allow a few minutes for a one-minute video.

Why does AI sometimes get it wrong?

Because some frames are ambiguous, even to a human: two people crossing, a blurred subject, the same color on subject and background. The model has to guess.

• Crossings: tracking can jump to the wrong person.
• Leaving the frame: the subject may be lost on return.
• Motion blur: edges become uncertain.
• Transparent objects: you see the background through them.

Why does processing happen on servers?

Because these models need powerful graphics cards, rarely found in a phone or a laptop. Running them locally would be slow, or impossible.

On Keyotte's servers, videos are only used to produce your export and are deleted 7 days after upload; see what happens to uploaded videos.

What does Keyotte do with the result?

The refined matte places the subject on flat green #00B140, and the video is encoded as an MP4 up to 1080p with the original sound. You can close the page during the render.

To use that file, see change a video background. To avoid mistakes at the shoot, how to film for background removal. And for the eternal question, green screen vs AI.