{"id":665,"date":"2026-04-04T17:03:26","date_gmt":"2026-04-04T09:03:26","guid":{"rendered":"https:\/\/connectword.dpdns.org\/?p=665"},"modified":"2026-04-04T17:03:26","modified_gmt":"2026-04-04T09:03:26","slug":"netflix-ai-team-just-open-sourced-void-an-ai-model-that-erases-objects-from-videos-physics-and-all","status":"publish","type":"post","link":"https:\/\/connectword.dpdns.org\/?p=665","title":{"rendered":"Netflix AI Team Just Open-Sourced VOID: an AI Model That Erases Objects From Videos \u2014 Physics and All"},"content":{"rendered":"

Video editing has always had a dirty secret: removing an object from footage is easy; making the scene look like it was never there is brutally hard. Take out a person holding a guitar, and you\u2019re left with a floating instrument that defies gravity. Hollywood VFX teams spend weeks fixing exactly this kind of problem. A team of researchers from Netflix and INSAIT, Sofia University \u2018St. Kliment Ohridski,\u2019 released VOID<\/strong> (Video Object and Interaction Deletion<\/strong>) model that can do it automatically.<\/p>\n

VOID removes objects from videos along with all interactions they induce on the scene \u2014 not just secondary effects like shadows and reflections, but physical interactions like objects falling when a person is removed.<\/p>\n

What Problem Is VOID Actually Solving?<\/strong><\/h3>\n

Standard video inpainting models \u2014 the kind used in most editing workflows today \u2014 are trained to fill in the pixel region where an object was. They\u2019re essentially very sophisticated background painters. What they don\u2019t do is reason about causality<\/em>: if I remove an actor who is holding a prop, what should happen to that prop?<\/p>\n

Existing video object removal methods excel at inpainting content \u2018behind\u2019 the object and correcting appearance-level artifacts such as shadows and reflections. However, when the removed object has more significant interactions, such as collisions with other objects, current models fail to correct them and produce implausible results.<\/p>\n

VOID is built on top of CogVideoX and fine-tuned for video inpainting with interaction-aware mask conditioning. The key innovation is in how the model understands the scene \u2014 not just \u2018what pixels should I fill?\u2019 but \u2018what is physically plausible after this object disappears?\u2019<\/p>\n

The canonical example from the research paper: if a person holding a guitar is removed, VOID also removes the person\u2019s effect on the guitar \u2014 causing it to fall naturally. That\u2019s not trivial. The model has to understand that the guitar was being supported<\/em> by the person, and that removing the person means gravity takes over.<\/p>\n

And unlike prior work, VOID was evaluated head-to-head against real competitors. Experiments on both synthetic and real data show that the approach better preserves consistent scene dynamics after object removal compared to prior video object removal methods including ProPainter, DiffuEraser, Runway, MiniMax-Remover, ROSE, and Gen-Omnimatte.<\/p>\n

\n
\"\"
https:\/\/arxiv.org\/pdf\/2604.02296<\/figcaption><\/figure>\n<\/div>\n

The Architecture: CogVideoX Under the Hood<\/strong><\/h3>\n

VOID is built on CogVideoX-Fun-V1.5-5b-InP<\/a> \u2014 a model from Alibaba PAI \u2014 and fine-tuned for video inpainting with interaction-aware quadmask<\/strong> conditioning. CogVideoX is a 3D Transformer-based video generation model. Think of it like a video version of Stable Diffusion \u2014 a diffusion model that operates over temporal sequences of frames rather than single images. The specific base model (CogVideoX-Fun-V1.5-5b-InP<\/code>) is released by Alibaba PAI on Hugging Face, which is the checkpoint engineers will need to download separately before running VOID.<\/p>\n

The fine-tuned architecture specs: a CogVideoX 3D Transformer with 5B parameters, taking video, quadmask, and a text prompt describing the scene after removal as input, operating at a default resolution of 384\u00d7672, processing a maximum of 197 frames, using the DDIM scheduler, and running in BF16 with FP8 quantization for memory efficiency.<\/p>\n

The quadmask<\/strong> is arguably the most interesting technical contribution here. Rather than a binary mask (remove this pixel \/ keep this pixel), the quadmask is a 4-value mask that encodes the primary object to remove, overlap regions, affected regions (falling objects, displaced items), and background to keep.<\/p>\n

In practice, each pixel in the mask gets one of four values: 0<\/code> (primary object being removed), 63<\/code> (overlap between primary and affected regions), 127<\/code> (interaction-affected region \u2014 things that will move or change as a result of the removal), and 255<\/code> (background, keep as-is). This gives the model a structured semantic map of what\u2019s happening in the scene<\/em>, not just where the object is<\/em>.<\/p>\n

Two-Pass Inference Pipeline<\/strong><\/h3>\n

VOID uses two transformer checkpoints, trained sequentially. You can run inference with Pass 1 alone or chain both passes for higher temporal consistency.<\/p>\n

Pass 1 (void_pass1.safetensors<\/code>) is the base inpainting model and is sufficient for most videos. Pass 2 serves a specific purpose: correcting a known failure mode. If the model detects object morphing \u2014 a known failure mode of smaller video diffusion models \u2014 an optional second pass re-runs inference using flow-warped noise derived from the first pass, stabilizing object shape along the newly synthesized trajectories.<\/p>\n

It\u2019s worth understanding the distinction: Pass 2 isn\u2019t just for longer clips \u2014 it\u2019s specifically a shape-stability fix<\/em>. When the diffusion model produces objects that gradually warp or deform across frames (a well-documented artifact in video diffusion), Pass 2 uses optical flow to warp the latents from Pass 1 and feeds them as initialization into a second diffusion run, anchoring the shape of synthesized objects frame-to-frame.<\/p>\n

How the Training Data Was Generated<\/strong><\/h3>\n

This is where things get genuinely interesting. Training a model to understand physical interactions requires paired videos \u2014 the same scene, with and without the object, where the physics plays out correctly in both. Real-world paired data at this scale doesn\u2019t exist. So the team built it synthetically.<\/p>\n

Training used paired counterfactual videos generated from two sources: HUMOTO \u2014 human-object interactions rendered in Blender with physics simulation \u2014 and Kubric \u2014 object-only interactions using Google Scanned Objects.<\/p>\n

HUMOTO uses motion-capture data of human-object interactions. The key mechanic is a Blender re-simulation: the scene is set up with a human and objects, rendered once with the human present, then the human is removed from the simulation and physics is re-run forward from that point. The result is a physically correct counterfactual \u2014 objects that were being held or supported now fall, exactly as they should. Kubric, developed by Google Research, applies the same idea to object-object collisions. Together, they produce a dataset of paired videos where the physics is provably correct, not approximated by a human annotator.<\/p>\n

Key Takeaways<\/strong><\/h3>\n