Nobody told this model what "object permanence" means. Nobody labeled a single frame "physically impossible." And yet, if you know exactly which layer to look in, part of the model already disagrees with what it's seeing — before it says a single word about it.
That's not a metaphor. It's a measurement, published quietly in February 2026, and almost nobody outside a small interpretability community noticed.
The bet nobody wanted to make out loud
For a couple of years, Yann LeCun — then Meta's chief AI scientist — had been making an argument that sounded almost like a concession: text-only language models, however fluent, don't actually understand the physical world, because they've never had to predict what happens to it. His proposed fix was JEPA — Joint Embedding Predictive Architecture — models that learn by watching video and predicting what comes next in an abstract space, not by generating pixels. The pitch was that something like physical intuition would show up on its own, the way it does in an infant who's never taken a physics class but still looks surprised when a ball seems to float.
In June 2025, Meta shipped V-JEPA 2 to back that bet up. Trained on over a million hours of web video plus a comparatively tiny 62 hours of real robot footage, it hit an 80% success rate on zero-shot robot control, against 15% for a comparable baseline. On IntPhys, a benchmark built specifically to test whether a model can tell possible from impossible physical events, it scored 98% zero-shot. Untrained, randomly initialized networks — and, tellingly, most pixel-generating video models — sat close to chance, around 50%.
That's a big number. It's also, by itself, not proof of much. A model can ace a benchmark by learning a shortcut that has nothing to do with the concept the benchmark is trying to measure. Around the same period, Google DeepMind and the INSAIT institute — led by PhD student Saman Motamed — published Physics-IQ, a broader benchmark testing physical reasoning across pixel-generating video models: Sora, Runway, Pika, Stable Video Diffusion, VideoPoet. The finding there was the opposite headline: visually convincing video, with what the authors called a striking lack of physical understanding underneath. Physics-IQ didn't test V-JEPA2 directly — it's a different category of model, predicting embeddings rather than pixels — but it made one thing clear going into 2026: "video models understand physics" was, as a blanket claim, contested and architecture-dependent. Nobody had actually opened up a model like V-JEPA2 and checked what was happening inside it.Somebody actually went and looked
Somebody actually went and looked
That's what changed in February. Sonia Joseph — a McGill/Mila PhD candidate who'd already built Prisma, an open-source toolkit for poking around inside vision and video transformers — and a team at Meta Superintelligence Labs did something simple to describe and hard to do: working across several video world model architectures, including V-JEPA2, they trained a linear probe at each layer to see if it could predict, from the model's internal activations alone, whether the video it was watching was physically possible.
Most layers, the probe couldn't tell. Then, in a narrow, specific band of mid-depth layers — roughly a third of the way into the network — it suddenly could, well above chance. The signal appeared, peaked, and then faded again as you moved toward the output. Joseph's team named this band the Physics Emergence Zone. Motion magnitude showed up early; the harder question of direction — which way something was actually moving, and whether that direction made physical sense — only became linearly readable right at that transition.
Put plainly: somewhere in the middle of these models, and nowhere else, there's a place where "does this look physically real" becomes something you can partially read off with a straight line through the model's internal space. Nobody trained it to build that. It just showed up. "Partially" is doing real work in that sentence, and it matters for what comes next.
Then someone asked the more dangerous question
Finding a signal is one thing. The next paper, published in May by Nahid Alam at Oreon Labs, working with the Cohere Labs Community, asked whether that signal was just sitting there being read, or whether it was actually doing something.
Alam's team didn't use V-JEPA2. They used VideoMAE-base, a smaller, 12-layer transformer video encoder — a different model than the one carrying this story's headline numbers, but one where Joseph's team had also found the same Physics Emergence Zone. Alam's probe peaked at layer 5 of 12, with an accuracy of 70.1% — chance is 50%. That's a real, above-chance signal. It is not, on its own, overwhelming.Here's what made it more than a readability result. Alam's team took the direction that probe had found and, at inference time, added a scaled version of it directly into the model's hidden states — no retraining, no fine-tuning, nothing touched except that one direction, turned up. Push it one way, and the model's confidence that a video is physically impossible saturates at 100%. Push it the other way, and that confidence drops to zero. It works in both directions, and it saturates fast — by an injection strength of just 5 (they tested up to 20), the effect is already maxed out.
Here's what made it more than a readability result. Alam's team took the direction that probe had found and, at inference time, added a scaled version of it directly into the model's hidden states — no retraining, no fine-tuning, nothing touched except that one direction, turned up. Push it one way, and the model's confidence that a video is physically impossible saturates at 100%. Push it the other way, and that confidence drops to zero. It works in both directions, and it saturates fast — by an injection strength of just 5 (they tested up to 20), the effect is already maxed out.
There's an asymmetry buried in the numbers worth sitting with. Steering toward "impossible" only flipped 25% of test videos — because the unsteered model was already calling roughly 75% of them impossible before anyone touched anything. Steering the other way, toward "possible," flipped 75%, cleaning up that same lopsided baseline. Read differently: before any intervention at all, this model's default judgment was tilted hard toward seeing things as physically wrong. That's either a sign the model is unusually cautious about physical violations, or a sign its calibration is off in a way the flip-rate headline alone doesn't show you.
A CVPR 2026 workshop version of Alam's paper circulated in June, and a separate group working on video diffusion models — a different architecture family again — found something similar in "The Invisible Hand of Physics": a physics signal that's decodable but never actually shows up in what the model generates. Which suggests this isn't a one-off quirk of any single training recipe. It might be a property of how these models learn from video in general.
The part that should make you slow down
Here's where I'd normally say the story escalates into "and that's why this matters" — except the honest version is messier than that, and the mess is the actual point.
A decade-old, well-cited line of interpretability research has been warning about exactly this setup. John Hewitt and Percy Liang showed back in 2019 that probing classifiers can hit high accuracy through pure memorization of their own training data — accuracy that tells you more about the probe than about what the model actually represents. Yonatan Belinkov's 2022 survey names the trap directly: information can be linearly decodable in a representation without the model's own downstream computation ever consulting it when it makes a prediction. Readable is not the same as used. Used is not the same as understood.Neither the Joseph nor the Alam paper reports the specific control-task test Hewitt and Liang proposed to rule out memorization. That's not an accusation of bad science — it's a gap, and a specific, checkable one. And a direct critique of the V-JEPA/IntPhys claims argues the benchmark's synthetic violation-of-expectation setup might reward pixel-motion shortcuts — trajectories that "look wrong" statistically — rather than anything resembling causal physical reasoning.
So hold all of it at once. A narrow, specific place inside these models — found independently in a JEPA-style model and a masked-autoencoder model — is where physical plausibility becomes linearly readable, at 70.1% accuracy against a 50% baseline. That same place can be used to causally steer the model's judgment, saturating completely in either direction by a modest injection strength. That's real, it replicated across model families, and it's not something you get by accident. And the exact kind of evidence being used to call this "understanding" is the exact kind of evidence the field's own methodologists have spent years warning can look like understanding without being it.
Key things worth holding onto
The Physics Emergence Zone is a real, replicated, mid-depth transition — found independently in a JEPA-style model (V-JEPA2) and a masked-autoencoder model (VideoMAE) — where physical plausibility becomes linearly decodable from internal activations, though the underlying probe accuracy (70.1% at its peak, vs. 50% chance) is meaningfully above chance without being conclusive.
It's not just readable, it's steerable: injecting the discovered direction back into VideoMAE's hidden states saturates the model's plausibility judgment completely in either direction by injection strength 5, with no retraining involved.
The flip-rate numbers are asymmetric (25% one way, 75% the other) because the unsteered model already classified about 75% of test videos as "impossible" before any intervention — a baseline-calibration detail easy to miss if you only quote the headline "steerable" result.
The same finding that makes this look like emergent physical intuition is also, structurally, the exact pattern that decade-old probing critiques (Hewitt & Liang, Belinkov) warn can be a memorization or shortcut artifact rather than genuine understanding — and neither current paper reports the control-task check that would rule that out.
Where that leaves it
I don't think this resolves cleanly, and I'm suspicious of anyone who tells you it does. A model that can be pushed, with one vector, from "impossible" to "100% plausible" and back either has a real, addressable physics representation sitting inside it — or it has a shortcut so shallow that a single line through its activation space is enough to break it. Both of those are consistent with the same experiment. Which one is actually happening probably depends on a control-task test nobody's published yet.
What would it take for you to believe a model actually understands something, rather than just reliably pattern-matching its way to the right answer — and would you know the difference if you saw it?
No comments:
Post a Comment