Induction Labs Photon-1 Simulates Desktops, Plays Checkers, and Models Billiard Physics From One Pretraining Run

By

Most agents that learn from video need to know what action produced each frame. Induction Labs is arguing that this requirement is the bottleneck. Last week, they released imagination models, a foundation model architecture that pretrains on raw video with no action labels at all.

Their test system is Photon-1, a sparse 106B-A5B mixture-of-experts (MoE) transformer trained on 18 years of computer demonstration video. On an internal computer use benchmark, Induction Labs reports that Photon-1 beats Gemini 3.1 Flash-Lite while using far less pretraining compute and costing roughly 3× less to serve.

What an imagination model actually does

An imagination model predicts future frames autoregressively using a next-latent-token-prediction objective. It does not generate pixels during pretraining. Everything is modeled in a learned representation space.

The claim that matters is this: predicting future states teaches the model to complete tasks, even though it never sees an action during pretraining. Induction Labs calls this an implicit policy. The model learns concepts of what a person is doing, rather than a label for each mouse click.

The compression trick that makes it scale

The architecture depends on a vision encoder that uses finite scalar quantization (FSQ). Each frame is compressed into 960 discrete tokens. Each token is an 8-dimensional vector. Each dimension takes one of five values: −1, −1/2, 0, 1/2, 1. That gives a codebook of 5⁸ possible codes.

The resulting encoding is about 2.2 KB per frame. Induction Labs reports over 100× better compression than existing OCR and multimodal-model representations, while preserving text, layout and state changes.

To hit that rate, Photon-1 uses a differential latent encoder. It encodes video frames as pairs, so the latents describe differences between frames rather than frame contents.

Data and pretraining compute

The corpus starts from an internal index of 2 billion publicly available videos. Filtering reduces that to roughly 2 million computer screen recordings. An internal keyframe detection model strips redundant frames.

The final dataset is 575 million frames, sampled at 1 frame per second. That equals 552 billion tokens, or about 18 years of video. Photon-1 was pretrained from scratch for a single epoch.

Training the 106B-A5B MoE at 32K context took approximately 30,000 H200 GPU-hours, or 4.4×10²² training FLOPs. The research team implemented training in PyTorch with custom fused kernels for the vision encoder and MoE layers, sustaining 40% end-to-end MFU. Those three figures are mutually consistent: 30,000 H200-hours at 40% MFU lands almost exactly on 4.3×10²².

From imagination to action

Induction Labs finetuned Photon-1 on fewer than 35,000 computer use trajectories to teach the action and instruction format. Special computer use tokens let the model emit actions. At inference, Photon-1 predicts the next frame’s state first, then outputs the action that gets there.

Online reinforcement learning follows. Rollouts run in real time on virtual machines at scale, and outcomes are verified programmatically to produce reward. The Linux VMs run five desktop environments (LXQt, Xfce, MATE, GNOME and Plasma), each with a Google account for login-restricted web apps and an internal ChatGPT clone with no rate limits.

The compute and cost comparison

ModelPretraining computeWeighted inference cost / 1M tokens*
Gemini 3.1 Flash-Lite1.200 × 10²⁴ FLOPs$0.36
Photon-10.044 × 10²⁴ FLOPs$0.11

*Weighted at a 10:1 input-to-output token ratio, which Induction Labs says matches its computer use tests.

Two caveats belong next to that table. First, the Gemini figure is Induction Labs’ own conservative estimate, assuming 8B active parameters and 25T pretraining tokens. Taken at face value the ratio is about 27×, not the 30× headline; Induction Labs states “at least 30×” on the basis that the true Gemini number is likely higher and the model was likely distilled. Second, the benchmark is internal and unreleased, so the result is not independently reproducible today.

Photon-1’s own breakeven cost on Induction Labs’ hardware is $0.06 per 1M input tokens and $0.60 per 1M output tokens, with no speculative decoding.

Does it generalize past the desktop?

This is the more interesting test, because Photon-1 saw only computer use video. The research team finetuned it on domains absent from pretraining and compared against two baselines: a vision encoder baseline with the same architecture and size but no imagination pretraining, and an LLM baseline (Ling-flash-2.0 from Inclusion AI, pretrained on 20T tokens).

On 20,000 tournament checkers games from the Open Checkers Archive 2.0, Photon-1 beat both baselines on world simulation and on move quality. On 10,000 synthetically generated billiard games simulated at 5 fps, it produced a mean absolute error of 0.47 against the ground-truth physics engine, versus 1.15 for the LLM baseline and 1.44 for the vision encoder baseline.

Photon-1 also picked up human priors from the pretraining video. After RL, it learned to use the in-VM ChatGPT clone to draft artifacts and answer knowledge questions, steering the LLM the way a person would.

Key Takeaways

  • Photon-1 learns an implicit policy from 18 years of screen recordings with zero action labels, using next-latent-token prediction.
  • FSQ compresses each frame to 960 tokens (~2.2 KB), a reported 100× gain over OCR and multimodal representations.
  • Trained for ~30,000 H200 GPU-hours, it beats Gemini 3.1 Flash-Lite on an internal benchmark at ~27× less pretraining compute.
  • Despite seeing only desktop video, it beats an LLM baseline at checkers and billiard physics after finetuning.
  • No weights, no API, no license — this is a research result, not a deployable model.

Check out the full technical writeup from Induction Labs and the announcement thread on X. All credit for this research goes to the researchers of this project.

The post Induction Labs Photon-1 Simulates Desktops, Plays Checkers, and Models Billiard Physics From One Pretraining Run appeared first on MarkTechPost.



from MarkTechPost https://ift.tt/S2QdbBI
via IFTTT

Comments

Post a Comment

Popular posts from this blog

Microsoft AI Proposes BitNet Distillation (BitDistill): A Lightweight Pipeline that Delivers up to 10x Memory Savings and about 2.65x CPU Speedup

By
Image
Microsoft Research proposes BitNet Distillation , a pipeline that converts existing full precision LLMs into 1.58 bit BitNet students for specific tasks, while keeping accuracy close to the FP16 teacher and improving CPU efficiency. The method combines SubLN based architectural refinement , continued pre training , and dual signal distillation from logits and multi head attention relations. Reported results show up to 10× memory savings and about 2.65× faster CPU inference , with task metrics comparable to FP16 across multiple sizes. What BitNet Distillation changes? The community already showed that BitNet b1.58 can match full precision quality when trained from scratch, but converting a pretrained FP16 model directly to 1.58 bit often loses accuracy, and the gap grows as model size increases. BitNet Distillation targets this conversion problem for practical downstream deployment. It is designed to preserve accuracy while delivering CPU friendly ternary weights with INT8 activa...
Read more

Technical Deep Dive: Automating LLM Agent Mastery for Any MCP Server with MCP- RL and ART

By
Image
Table of contents Introduction What Is MCP- RL? ART: The Agent Reinforcement Trainer Code Walkthrough: Specializing LLMs with MCP- RL Explanation: Under the Hood: How MCP- RL Generalizes Real-World Impact and Benchmarks Architectural Overview Practical Integration Summary Introduction Empowering large language models (LLMs) to fluidly interact with dynamic, real-world environments is a new frontier for AI engineering. The Model Context Protocol (MCP) specification offers a standardized gateway through which LLMs can interface with arbitrary external systems—APIs, file systems, databases, applications, or tools—without needing custom glue code or brittle prompt hacks each time. Still, leveraging such toolsets programmatically, with robust reasoning across multi-step tasks, remains a formidable challenge. This is where the recent combination of MCP- RL (a reinforcement learning loop targeting MCP servers) and the open-source ART (Agent Reinforcement Trainer) library ...
Read more

Google AI Releases LangExtract: An Open Source Python Library that Extracts Structured Data from Unstructured Text Documents

By
In today’s data-driven world, valuable insights are often buried in unstructured text—be it clinical notes, lengthy legal contracts, or customer feedback threads. Extracting meaningful, traceable information from these documents is both a technical and practical challenge.  Google AI’s new open-source Python library, LangExtract , is designed to address this gap directly, using LLMs like Gemini to deliver powerful, automated extraction with traceability and transparency at its core. Key Innovations of LangExtract 1. Declarative and Traceable Extraction LangExtract lets users define custom extraction tasks using natural language instructions and high-quality “few-shot” examples. This empowers developers and analysts to  specify exactly which entities, relationships, or facts to extract, and in what structure . Crucially, every extracted piece of information is  tied directly back to its source text —enabling validation, auditing, and end-to-end traceability. 2....
Read more