GLOW robot in black and silver with a chip visible through its transparent head
GLOW

From Phys­i­cal Expe­ri­ence to Auton­o­mous Exe­cu­tion

GLOW: A Generative Learning Framework for General-Purpose Embodied Intelligence

Learn­ing from Phys­i­cal Inter­ac­tion, Acting through Task Con­text

01

Introduction

Recent advances in general-purpose mul­ti­modal models have led to sub­stan­tial improve­ments in visual under­stand­ing, spa­tial rea­son­ing, task plan­ning, and com­plex instruc­tion fol­low­ing, pro­vid­ing increas­ingly pow­er­ful cog­ni­tive foun­da­tions for robot­ics.[1] For embod­ied intel­li­gence, how­ever, under­stand­ing a task is not the same as com­plet­ing it. A robot must ulti­mately act in the phys­i­cal world through con­tin­u­ous per­cep­tion, decision-making, and exe­cu­tion, while coping with vari­a­tions in objects and envi­ron­ments, exe­cu­tion errors, and unex­pected feed­back.[2][3]

This gives rise to a core chal­lenge that dis­tin­guishes embod­ied intel­li­gence from purely dig­i­tal intel­li­gence: how can gen­eral cog­ni­tive capa­bil­i­ties be trans­lated into reli­able phys­i­cal action?

Phys­i­cal tasks require models to learn the rela­tion­ships among obser­va­tions, states, actions, and out­comes. A robot must not only iden­tify tar­gets, under­stand spa­tial rela­tion­ships, and plan sequences of oper­a­tions, but also deter­mine whether an action is fea­si­ble, how it will change the envi­ron­ment, and whether the result­ing state matches expec­ta­tions. When a task con­sists of mul­ti­ple inter­de­pen­dent steps, the model must also main­tain task state over time and adapt sub­se­quent deci­sions based on new obser­va­tions and exe­cu­tion feed­back. Embod­ied capa­bil­i­ties there­fore cannot be derived from lan­guage and visual knowl­edge alone. They must also be learned from real or syn­thetic phys­i­cal inter­ac­tion expe­ri­ence,[4][5] enabling models to cap­ture how the world responds to actions and to turn that expe­ri­ence into exe­cutable behav­ior.

To address this chal­lenge, Knowin intro­duces GLOW (Gen­er­a­tive Learn­ing of World), a gen­er­a­tive learn­ing frame­work for general-purpose embod­ied intel­li­gence. GLOW is built on real and syn­thetic phys­i­cal inter­ac­tion expe­ri­ence, uses world roll­outs to broaden the model’s expo­sure to pos­si­ble action out­comes, and adopts a uni­fied mul­ti­modal autore­gres­sive for­mu­la­tion that con­nects scene under­stand­ing, phys­i­cal rea­son­ing, task plan­ning, and action gen­er­a­tion. Closed-loop exe­cu­tion then turns model deci­sions into phys­i­cal robot actions. Within GLOW, expe­ri­ence con­struc­tion, world under­stand­ing, model train­ing, action gen­er­a­tion, and exe­cu­tion feed­back together form a learn­ing loop for acquir­ing phys­i­cal inter­ac­tion capa­bil­i­ties, pro­vid­ing a foun­da­tion for task exe­cu­tion across objects and envi­ron­ments.

The goal of GLOW is not to learn a sep­a­rate policy for every task, but to acquire embod­ied capa­bil­i­ties that can be reused across objects, envi­ron­ments, and tasks. The model first devel­ops foun­da­tional capa­bil­i­ties from phys­i­cal inter­ac­tion expe­ri­ence. When pre­sented with a new task, it then com­bines cur­rent obser­va­tions, task instruc­tions, demon­stra­tions, and other con­tex­tual infor­ma­tion to invoke, com­pose, and adapt these capa­bil­i­ties, enabling exe­cu­tion under new task and envi­ron­men­tal con­di­tions.

Exper­i­men­tal results show that GLOW not only under­stands phys­i­cal scenes, but also trans­lates that under­stand­ing into effec­tive task exe­cu­tion. Under the eval­u­a­tion set­tings used in this report, GLOW achieves an aver­age suc­cess rate of 62.2% across 18 RoboDojo tasks, out­per­form­ing the GPT-6 (Robocurve) base­line by 40.0 per­cent­age points. On LIBERO-Pro, GLOW achieves an aver­age suc­cess rate of 86.7%, exceed­ing the cor­re­spond­ing GPT-6 Astra base­line by 28.5 per­cent­age points. For embod­ied scene under­stand­ing, KnowinBrain-1.5 ranks No. 1 on the 2D embod­ied question-answering leader­board in Embod­ied Arena. Together, these results demon­strate GLOW’s capa­bil­i­ties across spa­tial under­stand­ing, phys­i­cal inter­ac­tion, and task exe­cu­tion.

Build­ing on these results, this report presents the over­all archi­tec­ture of GLOW, includ­ing phys­i­cal expe­ri­ence con­struc­tion, world roll­outs, mul­ti­modal autore­gres­sive mod­el­ing, and closed-loop exe­cu­tion. Through real-robot exper­i­ments, we fur­ther inves­ti­gate how the model can orga­nize and reuse exist­ing capa­bil­i­ties based on task con­text. We also demon­strate real-world cases in which the model adapts to a new task from a single demon­stra­tion with­out updat­ing its param­e­ters, explor­ing a path from auton­o­mous exe­cu­tion toward in-context adap­ta­tion and one-shot teach­ing.

02

GLOW Architecture: From Experience Generation to Closed-Loop Execution

GLOW is built around KnowinGLOW, a uni­fied mul­ti­modal autore­gres­sive model that inte­grates the former Brain and Act capa­bil­i­ties as BrainSkill and ActSkill within a single model. It jointly learns visual under­stand­ing, spa­tial rea­son­ing, task plan­ning, and action gen­er­a­tion. These capa­bil­i­ties are aligned within a shared rep­re­sen­ta­tion and train­ing frame­work, con­nect­ing scene under­stand­ing, task deci­sions, and phys­i­cal actions to improve embod­ied task exe­cu­tion. KnowinDream pro­vides phys­i­cal inter­ac­tion expe­ri­ence for train­ing; KnowinWorld sup­ports action-outcome roll­outs and embod­ied align­ment; and KnowinAgent (Har­ness) man­ages task memory, tool calls, and exe­cu­tion feed­back. Together, they form an inte­grated system from experience-based learn­ing to closed-loop action.

GLOW System Architecture
KnowinDream generates embodied training experience; KnowinGLOW unifies understanding, reasoning, and action generation; KnowinWorld provides interactive world environments and execution feedback; and KnowinAgent manages memory, replanning, and skill accumulation. Together, they support experience-based learning, closed-loop action, and capability development.

Synthetic Embodied Experience: KnowinDream + KnowinWorld

KnowinDream builds every­day house­hold worlds, KnowinWorld rolls out the con­se­quences of actions within them, and the Expe­ri­ence Com­piler turns both into re­use­able expe­ri­ences.

KnowinDream: Generating Realistic Household Life

KnowinDream turns family roles, habits, and home seman­tics into phys­i­cal­ly grounded, pho­to­real house­hold inter­ac­tions.

KnowinDream runs three engines in sequence.

KnowinDream · Experience Generation and Compilation
Household programs become executable physical worlds, producing interaction trajectories and visual experiences. Experience Compiler aligns observations, actions, states, contacts, and other training information.

The Life Engine com­piles house­hold roles, habits, and the seman­tic struc­ture of the home into a House­hold Pro­gram H, which defines the envi­ron­ment, its oc­cu­pants, their rou­tines and tasks, the cur­rent state, and rel­e­vant events and con­straints. It sim­u­lates events as they unfold over time while main­tain­ing two distinct records: the actual state of the home and the robot’s obser­va­tion his­tory. For exam­ple, when a family member moves a cup outside the robot’s field of view, the world state up­dates imme­di­ately, but the robot’s last recorded obser­va­tion still places the cup at its pre­vi­ous loca­tion. Each expe­ri­ence records event timing, vis­i­bil­ity, and obser­va­tion sources, help­ing the model learn to assess whether past obser­va­tions remain valid and deter­mine when and where to seek up­dat­ed infor­ma­tion.

The Physics Engine turns the House­hold Pro­gram H pro­duced by Life Engine into an exe­cutable 3D envi­ron­ment and sim­u­lates how actions and events change that envi­ron­ment over time. Every object in H becomes a 3D body with geom­e­try, phys­i­cal prop­er­ties, and joints, and every action has to be phys­i­cal­ly pos­si­ble: “put the cup in the cab­i­net” proceeds only if the cup is reach­able, the door is open, and the grasp holds. The result­ing motion and con­tacts come from physics sim­u­la­tion rather than being written into the pro­gram, and their effects persist—an opened door stays open and limits what the robot can reach next. The output is a Grounded Tra­jec­to­ry τ: a time-aligned record of actions, con­tacts, and object states in which every step is phys­i­cal­ly fea­si­ble and every label is read di­rect­ly from the sim­u­la­tion, so it is cor­rect by con­struc­tion.

The Dream Engine renders Pho­to­real Expe­ri­ence X around τ and varies it in appear­ance (light­ing, mate­ri­als, back­grounds), object (struc­turally com­pat­i­ble sub­sti­tutes, re-executed in the Physics Engine), and scene (home, camera view, robot con­fig­u­ra­tion), while the inter­ac­tion itself stays the same. The model thus sees one manip­u­la­tion rela­tion­ship across many appear­ances, objects, and view­points.

KnowinDream · Generating Worlds, Objects, and Views
From light­ing and inte­rior style to object sub­sti­tu­tion and wrist-camera views, gen­er­ated expe­ri­ences unfold through scene edit­ing and grasp­ing inter­ac­tions.

KnowinWorld: Action-Conditioned World Rollouts

Under­stand­ing the world includes antic­i­pat­ing the con­se­quences of actions. Research on mental sim­u­la­tion explains human judg­ments about scene changes through approx­i­mate phys­i­cal roll­outs. Start­ing from the cur­rent state, KnowinWorld pre­dicts sub­se­quent changes con­di­tioned on can­di­date actions, con­nect­ing cur­rent obser­va­tions to pos­si­ble con­tacts, motions, and state tran­si­tions. Mul­ti­ple views, robot state, task con­text, and 3D struc­ture jointly form a task-relevant spa­tial latent state zt. This rep­re­sen­ta­tion cap­tures spa­tial rela­tion­ships among objects, the robot’s loca­tion, and the goal cur­rently being pur­sued.

Con­di­tional dynam­ics applies can­di­date action sequences to the spa­tial latent state and rolls out future states under con­straints such as col­li­sion, con­tact, reach­a­bil­ity, and object per­ma­nence. Pick­ing up a cup there­fore con­sists of a con­tin­u­ous sequence of approach, con­tact, grasp, and lift. The state pro­duced at each stage becomes the start­ing point of the next.

01 / CONDITIONAL DYNAMICS
Conditional Dynamics and Candidate Evaluation
Z^t+1:t+H(j)=Fφ(zt,A(j),Cphys,b)
j⋆∈arg maxj∈{1,…,J}S(Z^t+1:t+H(j),g)

Here, zt is the current spatial latent state; A(j) = (at(j), …, at+H−1(j)) is candidate action sequence j, containing H actions and corresponding predicted states from t+1 to t+H; Cphys and b denote physical constraints and embodiment conditions, respectively. Fφ predicts the state sequence Ẑ over the next H steps, with dynamics-model parameters φ. S scores predictions against task goal g, and j⋆ identifies the highest-scoring branch among the J current candidates.

From a shared initial state and task goal, the model compares contact, displacement, and task progress along candidate paths to select subsequent actions.

Vary­ing the action from the same start­ing point pro­duces com­pa­ra­ble future branches. When pick­ing up a cup from a table, dif­fer­ent approaches and con­tact pat­terns may cause col­li­sions with nearby objects, tip the cup, or com­plete the grasp. A world-outcome eval­u­a­tor ranks can­di­date branches by task progress, phys­i­cal con­sis­tency, reach­a­bil­ity, risk, and pre­dic­tion uncer­tainty, pro­vid­ing their key con­se­quences to the deci­sion layer.

These branches also expand train­ing expe­ri­ence. Suc­cess­ful actions, fail­ures, and recov­ery sequences share the same ini­tial state while pre­serv­ing dif­fer­ent action-outcome rela­tion­ships. Dream pro­vides diverse worlds and inter­ac­tions, while World fur­ther unfolds their con­se­quences. Together, they supply data for learn­ing phys­i­cal causal­ity and task deci­sions.

KnowinWorld · Action-Conditioned World Rollouts
Starting from a shared spatial latent state, the model rolls out the futures of different candidate actions, compares paths against task goals and physical conditions, and aligns predictions with new observations.

Experience Compiler: Compiling Physical Experience into Supervision

Expe­ri­ence Com­piler con­verts visual obser­va­tions, action tra­jec­to­ries, world states, vis­i­bil­ity, and out­come records into train­ing sam­ples. Observation-action pairs sup­port behav­ior gen­er­a­tion; state-action-outcome tuples sup­port world pre­dic­tion; and com­plete tra­jec­to­ries with out­comes sup­port task-progress assess­ment. Dif­fer­ent super­vi­sion sig­nals share object iden­ti­ties, timing, and inter­ac­tion con­text from the same expe­ri­ence.

Grounded Expe­ri­ence pro­vides actions, con­tacts, and object states from phys­i­cal inter­ac­tions. Anchored Gen­er­a­tive Expe­ri­ence expands visual and seman­tic con­di­tions around those inter­ac­tions. Dreamed Expe­ri­ence unfolds can­di­date futures and their out­comes. Expe­ri­ence com­pi­la­tion orga­nizes these sources into model inputs and out­puts, directly con­nect­ing data gen­er­a­tion to spe­cific learn­ing objec­tives.

02 / EXPERIENCE COMPILATION
Experience Compilation and Supervision
Dk=⋃e∈ECompilek(τe,Xe)

Here, ℰ denotes the collection of experiences; τe records the physical trajectory and world state of experience e, while Xe records the corresponding visual experience. Compilek produces training samples for supervision type k, which are combined into dataset 𝒟k. Types k may include action generation, future prediction, state understanding, or outcome assessment.

Compilation changes how inputs and targets are organized while preserving object identities, temporal alignment, and physical processes. A single interaction can therefore support multiple training objectives.

Information and Learning Objectives Within an InteractionEXPERIENCE COMPILER
ExperienceInput–Target MappingSupported Capability
Observations and actionsCur­rent image, task, and robot state → end-effector action / grip­per con­trolGen­er­ate the next action from cur­rent con­di­tions
States and outcomesCur­rent world state + can­di­date action → sub­se­quent state changesAction-conditioned future pre­dic­tion and phys­i­cal rea­son­ing
Events and visibilityObserved events, times, and sources → state and belief updatesTem­po­ral under­stand­ing and task-information orga­ni­za­tion
Trajectories and resultsCon­tin­u­ous inter­ac­tions → task progress, out­comes, and path eval­u­a­tionStage assess­ment, out­come ver­i­fi­ca­tion, and can­di­date com­par­i­son
Adjustment and recoveryStates encoun­tered during inter­ac­tion → cor­rec­tive actions and sub­se­quent stepsCon­tinue and replan based on feed­back

KnowinWorld in Action: Dual-Arm Simulated Tasks

Tube inser­tion, bowl stack­ing, and block stack­ing illus­trate KnowinWorld’s dual-arm sim­u­la­tion tasks. The arms switch roles during approach, align­ment, inser­tion, han­dover, and place­ment. The manip­u­la­tion arm advances the inter­ac­tion, while the obser­va­tion arm changes view­point to cap­ture spa­tial rela­tion­ships between objects and tar­gets. Together, they gen­er­ate con­tin­u­ous inter­ac­tion expe­ri­ence.

During local manip­u­la­tion, RGB-D obser­va­tions and selected pixel posi­tions pro­vide geo­met­ric ref­er­ences; wrist-camera motion adds con­tact and align­ment infor­ma­tion. The task layer sets stage goals, while the inter­ac­tion layer acts on cur­rent geom­e­try and returns results. New obser­va­tions between actions keep sub­se­quent deci­sions aligned with the scene.

TWO VIEWS / ONE TASK
Two Views of the Same Interaction

The video follows a robot as it grasps a bottle lying on the table, rotates it upright, and sets it down. The wider scene view shows the bottle in rela­tion to surrounding objects, while the close-up view reveals the grip­per’s con­tact and motion. Together, the two views con­nect phys­i­cal inter­ac­tion with the chang­ing state of the scene through­out the task.

KnowinWorld · Dual-arm sim­u­lated inter­ac­tion. Scenes, tasks, mul­ti­ple views, and action out­comes form con­tin­u­ous embod­ied expe­ri­ence.

KnowinWorld builds inter­ac­tive vir­tual worlds with high-fidelity phys­i­cal inter­ac­tions and visual appear­ance. Together with KnowinDream, it sup­ports high-fidelity scene ran­dom­iza­tion, broad­en­ing the dis­tri­bu­tion of train­ing data across visual, spa­tial, and physical-interaction con­di­tions. This pro­vides data for mul­ti­di­men­sional dis­tri­bu­tion gen­er­al­iza­tion in embod­ied foun­da­tion models. The fol­low­ing bowl-stacking task illus­trates this data-generation approach.

Stacking bowls

KnowinDream supplies the household scenes and events, KnowinWorld supplies the consequences, and the Experience Compiler consolidates both into training experience for observation understanding, spatial reasoning, task planning, and action generation.

A Unified Autoregressive Embodied Model

Model obser­va­tions, tasks, his­tory, and actions in a uni­fied autore­gres­sive sequence.

Multimodal Task Context

GLOW’s model core uses a native autore­gres­sive archi­tec­ture. Images and videos from robot cam­eras, user lan­guage, robot state, and exe­cu­tion feed­back jointly form its inputs. Con­di­tioned on the cur­rent task and progress, the model gen­er­ates sub­se­quent judg­ments and actions. Real first-person expe­ri­ence and syn­thetic expe­ri­ence orga­nized by KnowinDream and KnowinWorld are used together in train­ing to con­nect scene under­stand­ing, phys­i­cal rea­son­ing, and task deci­sions.

Vision iden­ti­fies objects and states; lan­guage spec­i­fies goals and rules; pro­pri­o­cep­tion describes exe­cu­tion con­di­tions; and his­tory retains actions and out­comes. The model com­bines this infor­ma­tion to gen­er­ate the cur­rent action. As the task advances, new obser­va­tions and feed­back enter the con­text and drive the next deci­sion.

KnowinGLOW · From Context to Action
Vision, task information, and robot state enter the model context. KnowinGLOW directly generates EEF actions and gripper control, which the motion interface executes on the robot. Execution results become subsequent inputs.

Autoregressive Modeling and Sequence Supervision

Autore­gres­sive mod­el­ing learns how suc­ces­sive out­puts depend on their pre­ced­ing con­text. Cur­rent obser­va­tions and task infor­ma­tion form a con­di­tion­ing prefix, and newly gen­er­ated out­puts are appended to the same sequence. Object local­iza­tion iden­ti­fies the enti­ties needed for rela­tional judg­ments; spa­tial and phys­i­cal rea­son­ing guide action selec­tion; and action out­comes inform the next round of rea­son­ing. Mul­ti­ple deci­sions within a task there­fore remain con­nected to a shared goal.

03 / SEQUENCE MODELING
Autoregressive Modeling and the Sequence Supervision Objective
pθ(Y∣C)=∏i=1Npθ(yi∣C,y<i)
ℓseq(θ;C,Y)=−logpθ(Y∣C)=−∑i=1Nlogpθ(yi∣C,y<i)

Here, C is the context comprising observations, task information, proprioceptive state, and history; Y = (y₁, …, yN) is the target token sequence, including an end token; y<i denotes the target tokens before the current position; θ denotes model parameters. ℓseq is the negative log-likelihood of one target sequence, used for sequence-supervised training.

The probability factorization expresses each output’s dependence on context, while negative log-likelihood expresses sequence supervision as a sum of per-token losses. Training and inference share the same conditional formulation.

Train­ing sam­ples pro­vide spe­cific super­vi­sion for these con­di­tional rela­tion­ships. Object-and-attribute sam­ples con­nect visual regions to seman­tics; spa­tial sam­ples con­nect ref­er­ence objects to rel­a­tive rela­tion­ships; inter­ac­tion sam­ples con­nect actions to state changes; and plan­ning sam­ples con­nect the cur­rent stage to the next step. Expe­ri­ence com­pi­la­tion pre­serves object and tem­po­ral cor­re­spon­dences across these sam­ples, allow­ing dif­fer­ent super­vi­sion sig­nals to serve the same embod­ied task inter­face.

Multi-Capability Pretraining and Spatial–Action Alignment

Pre­train­ing spans diverse house­hold envi­ron­ments and com­bines real first-person expe­ri­ence with syn­thetic expe­ri­ence. It learns object iden­ti­ties, ref­er­ence rela­tion­ships, reach­able regions, action pro­cesses, and scene changes across spa­tial lay­outs, view­points, object appear­ances, and place­ments. Multi-stage pre­train­ing orga­nizes super­vi­sion for spa­tial local­iza­tion, rela­tional judg­ment, state under­stand­ing, and task plan­ning. This estab­lishes rela­tion­ships among objects, ref­er­ence enti­ties, target regions, and the robot’s body, devel­op­ing trans­fer­able spa­tial under­stand­ing of house­hold scenes.

Post-training first strength­ens and aligns spa­tial under­stand­ing, task plan­ning, and action exe­cu­tion sep­a­rately through reinforcement-learning explo­ration, using task feed­back to rein­force rea­son­ing and goal-directed behav­ior. MOPD then inte­grates these enhanced capa­bil­i­ties within a uni­fied opti­miza­tion process, fur­ther coor­di­nat­ing spa­tial judg­ment, action gen­er­a­tion, and exe­cu­tion feed­back so that under­stand­ing, plan­ning, and exe­cu­tion serve the same task.

This align­ment spans task stages. Approach empha­sizes target posi­tion and reach­able paths; grasp­ing empha­sizes the grasp rela­tion­ship and object response; trans­port empha­sizes the object’s posi­tion rel­a­tive to the target region; and con­tact empha­sizes action-induced state changes. Each stage links its actions to scene con­di­tions and out­comes, form­ing a task process that sup­ports con­tin­ued rea­son­ing.

Connecting Experience, Supervision, and Model CapabilitiesTRAINING ROUTE
StageData and Model ProcessingResulting Task Capabilities
Experience organizationAlign obser­va­tions, states, actions, and out­comes across real first-person data and syn­thetic expe­ri­ence.Main­tain con­sis­tent object iden­ti­ties, timing, and inter­ac­tion con­text.
PretrainingLearn objects, rela­tion­ships, actions, and scene pro­cesses from visual and task expe­ri­ence.First-person under­stand­ing, spa­tial rela­tion­ships, and phys­i­cal inter­ac­tion under­stand­ing.
Spatial–action post-trainingEnhance and align capa­bil­i­ties sep­a­rately through reinforcement-learning explo­ration, then inte­grate them through MOPD.Con­nect scene judg­ments to goal-directed action rep­re­sen­ta­tions.
Execution-interface alignmentUnify coor­di­nates, units, and exe­cu­tion bounds for end-effector pose and grip­per con­trol.Con­nect model-generated EEF actions to robot motion exe­cu­tion.

Learning Relationships Across Appearances, Objects, and Views

Syn­thetic expe­ri­ence pro­vides sys­tem­atic vari­a­tion in con­di­tions. Appear­ance expan­sion places the same inter­ac­tion rela­tion­ship in dif­fer­ent visual envi­ron­ments; object sub­sti­tu­tion exposes the model to dif­fer­ent geome­tries and affor­dances; world-configuration changes con­nect view­points, embod­i­ments, and spa­tial lay­outs; and can­di­date futures com­pare action choices and out­comes from a common start­ing point. Together, these sam­ples help the model dis­tin­guish invari­ant task rela­tion­ships from chang­ing con­di­tions.

The uni­fied model learns rela­tion­ships among objects, space, actions, and goals. For a new task, it com­bines task infor­ma­tion, cur­rent obser­va­tions, and inter­ac­tion his­tory to gen­er­ate actions, then con­tin­ues rea­son­ing and plan­ning from exe­cu­tion results.

Embodied In-Context Learning in GLOW

Teach­ing pro­vides task rules, cur­rent obser­va­tions pro­vide exe­cu­tion con­di­tions, and the model con­nects the two.[6]

GLOW incor­po­rates task descrip­tions, demon­strated object rela­tion­ships, oper­a­tion order, and target states into run­time con­text. The model inter­prets the teach­ing, maps demon­strated rela­tion­ships to the cur­rent scene, and gen­er­ates sub­se­quent judg­ments and actions. User teach­ing thus directly informs task exe­cu­tion.

Embod­ied teach­ing spec­i­fies both task goals and com­ple­tion con­di­tions: object ref­er­ences iden­tify what to manip­u­late, spa­tial rela­tion­ships spec­ify target regions, step depen­den­cies estab­lish oper­a­tion order, and target states define com­ple­tion cri­te­ria.[7] The model uses this infor­ma­tion to under­stand the task and updates progress from new obser­va­tions and exe­cu­tion feed­back.

Mapping Objects and Relationships Across Scenes

GLOW uses first-person under­stand­ing to iden­tify objects, attributes, and actions, and spa­tiotem­po­ral local­iza­tion to deter­mine target regions, ref­er­ence points, and event order. These capa­bil­i­ties map demon­strated objects and rela­tion­ships to the cur­rent envi­ron­ment.

When exe­cut­ing “place the object in the target con­tainer,” the model com­bines the rel­a­tive posi­tions of the car­ried object, con­tainer, and robot end effec­tor. Cur­rent views and on-demand spa­tial queries deter­mine how to approach and align. The demon­strated task rela­tion­ship thereby becomes a con­crete oper­a­tion.

Task Decomposition and Stage Progression

The sequence in a demon­stra­tion pro­vides task struc­ture. Approach, grasp, lift, trans­port, and align­ment depend on dif­fer­ent scene con­di­tions and pro­duce dif­fer­ent state changes. The model uses cur­rent obser­va­tions to iden­tify the task stage, con­nects demon­strated steps to progress already made, and gen­er­ates the next step.

Goal-directed plan­ning bridges these stages: it works back­ward from the target state to select actions that can advance the task under cur­rent con­di­tions. After exe­cu­tion, new object states and inter­ac­tion out­comes deter­mine whether to pro­ceed to the next stage. Teach­ing pro­vides over­all direc­tion, task his­tory main­tains con­ti­nu­ity, and real-time obser­va­tions pro­vide cur­rent evi­dence.

Teaching Context and Physical Feedback

Object posi­tion after grip­per clo­sure, whether an object moves with the grip­per during a test lift, and drawer response to an out­ward pull con­tin­u­ously reflect task progress. Har­ness incor­po­rates this infor­ma­tion into exe­cu­tion receipts so that the next deci­sion can use the orig­i­nal goal, teach­ing con­text, and latest inter­ac­tion results together.

One-shot teach­ing spans task under­stand­ing and exe­cu­tion: the user expresses intent through teach­ing, the model maps the demon­stra­tion to the scene, the exe­cu­tion system car­ries out actions, and feed­back updates deci­sions. Start­ing from one teach­ing exam­ple, the robot com­pletes the task in its cur­rent envi­ron­ment.

From Teaching Content to Task ExecutionTEACHING → ACTION
Teaching InformationRole in the Current ContextConnected Capabilities
Objects and referencesMatch demon­strated objects to tar­gets in the cur­rent view.Object under­stand­ing, attribute recog­ni­tion, and region/reference-point local­iza­tion
Spatial and goal relationshipsUse cur­rent spa­tial rela­tion­ships to orga­nize approach, trans­port, and align­ment.Spa­tial rela­tion­ships, free space, and multi-view phys­i­cal rea­son­ing
Steps and operation orderCon­nect com­pleted stages to sub­se­quent actions.Action under­stand­ing, next-step pre­dic­tion, and multi-step plan­ning
Completion conditionsAssess progress from post-action object states and task out­comes.Interaction-outcome rea­son­ing, suc­cess ver­i­fi­ca­tion, and progress assess­ment

Closed-Loop Execution of Long-Horizon Tasks

Action rep­re­sen­ta­tions con­nect the model to con­trol. Har­ness keeps obser­va­tions, tools, memory, and feed­back aligned with a shared goal.

KnowinGLOW: Direct EEF Action Generation

KnowinGLOW con­nects model out­puts to robot motion exe­cu­tion. Con­di­tioned on images, task goals, robot state, and inter­ac­tion his­tory, it directly gen­er­ates end-effector (EEF) poses or pose incre­ments, along with grip­per con­trol. The motion inter­face exe­cutes requests and returns new obser­va­tions, pro­pri­o­cep­tive states, and results to con­di­tion the next action.

Actions pre­serve tem­po­ral struc­ture and stage rela­tion­ships. Fill­ing a cup, for exam­ple, involves reach­ing for the cup, grasp­ing and trans­port­ing it, align­ing and plac­ing it, and press­ing the dis­penser. The object and end-effector states pro­duced by each stage deter­mine the exe­cu­tion con­di­tions of the next. Action rep­re­sen­ta­tions are linked to these motion pro­cesses, ground­ing task rea­son­ing in spe­cific move­ments, poses, and con­tacts.

KnowinGLOW · Comparing Technical Paradigms
04 / EEF ACTION INTERFACE
Context-Conditioned End-Effector Action Generation
at∼πθ(⋅∣Ct),at=(ξt,ct)

Here, Ct is the current task context, πθ is the action policy, ct is gripper control, and ξt denotes end-effector action parameters. Pose requests specify target position and orientation; incremental requests specify relative translation and rotation. The corresponding action interface standardizes reference frames, rotation representations, and units.

The model uses pθ to generate token sequences autoregressively. Action parsing converts them into end-effector commands associated with policy πθ . New observations and results from each execution enter the next context and drive subsequent actions.

Cup grasp­ing and trans­port focus on object pose, end-effector ori­en­ta­tion, and the grasp rela­tion­ship. Align­ment and place­ment focus on the cup’s offset from its target, while press­ing focuses on button state and device changes after con­tact. New visual obser­va­tions, pro­pri­o­cep­tive states, and action feed­back re-enter the con­text to guide sub­se­quent gen­er­a­tion. The task advances through con­nected end-effector actions.

KnowinAgent（Harness）：A Context-Driven Task Runtime

Task con­text deter­mines how exist­ing capa­bil­i­ties are invoked and orga­nized. Cognitive-control theory empha­sizes main­tain­ing goals and the means to achieve them in order to orga­nize per­cep­tion, inter­nal states, and behav­ioral out­puts. Har­ness con­nects goals, obser­va­tions, memory, tools, and feed­back within a single robot run­time. Each rea­son­ing step can access the orig­i­nal goal, cur­rent exter­nal and wrist-camera images, robot state, recent tool results, and task progress. The model selects what infor­ma­tion to obtain, what actions to take, and which out­comes jus­tify con­tin­u­ing; the run­time han­dles calls, exe­cu­tion, and log­ging.

Goals guide tool selec­tion and action plan­ning; memory retains task stages and progress; tools pro­vide the mea­sure­ments and exe­cu­tion inter­faces needed at each moment; and feed­back informs later judg­ments. Bottle necks, han­dles, and con­tainer regions iden­ti­fied vis­ually become spa­tial queries, pro­vid­ing geo­met­ric ref­er­ences for approach and align­ment. After motion exe­cu­tion, actual dis­place­ment, grip­per state, and new camera images are returned together. Each tool call pre­serves the con­nec­tion from task intent to phys­i­cal out­come, keep­ing suc­ces­sive inter­ac­tions aligned with the same goal.

Harness · Long-Horizon Task Runtime
Reasoning, execution, feedback, and recovery form a continuous task chain. Persistent memory and dual-arm collaboration connect successive interactions to the same goal.

On-Demand Spatial Queries and Adaptive Action Granularity

Har­ness sup­ports spa­tial queries for regions and points, rel­a­tive geom­e­try, selected-feature track­ing, and can­di­date gripper-pose checks. The model chooses query tar­gets for the cur­rent stage, con­vert­ing vis­ually iden­ti­fied objects into spa­tial ref­er­ences for exe­cu­tion. Rela­tion­ships between car­ried objects and target con­tain­ers guide align­ment near the end of trans­port.

Action gran­u­lar­ity adapts to the inter­ac­tion stage. In open space, move_toward del­e­gates longer-distance motion to the local con­troller. Near a bottle neck, bowl rim, or drawer handle, the model uses bounded small move­ments and observes again between adjust­ments. The same task con­text con­nects long-distance motion to fine local manip­u­la­tion.

Spatial and Execution Interfaces Available to the ModelROBOT INTERFACES
InterfaceInputReturn Value and Purpose
Region / point spatial querySelected regions or points in exter­nal or wrist RGB-D views.Returns visible-surface geom­e­try as a manip­u­la­tion ref­er­ence.
Relative geometry querySelected ref­er­ence posi­tions on car­ried objects, target con­tain­ers, and other enti­ties.Returns rel­a­tive geom­e­try for approach and align­ment after trans­port.
Tracking and pose checksSelected fea­tures and can­di­date grip­per poses.Main­tains target cor­re­spon­dence and informs approach direc­tion and ori­en­ta­tion.
move_towardA motion target selected by the model.The local con­troller exe­cutes motion and returns motion records and new obser­va­tions.
Small movements / gripper operationsRel­a­tive dis­place­ment or a grip­per oper­a­tion.Returns actual dis­place­ment, grip­per state, and post-action obser­va­tions.
History retrieval / progress loggingRel­e­vant his­tory, cur­rent stage, mile­stones, or evi­dence ref­er­ences.Con­tin­u­ously orga­nizes actions and out­comes through­out the task.
Task completion checkCur­rent sim­u­la­tion state and a com­ple­tion request.Returns the result of check­ing task-success con­di­tions.

Execution Receipts and Closed-Loop Decisions

Exe­cu­tion receipts record actual dis­place­ment, remain­ing target error, grip­per state, and new obser­va­tions. The model com­pares requests with results to assess progress, then decides whether to con­tinue approach­ing, change ori­en­ta­tion, or advance to the next stage.

Short test lifts and pulls both advance the task and verify con­tact rela­tion­ships. A test lift checks the grasp by observ­ing whether the object moves with the grip­per; a test pull checks engage­ment through the drawer’s response. These obser­va­tions directly inform the next action.

05 / EXECUTION FEEDBACK
Executed Displacement and Feedback Residual
Δptactual=pt+1−pt
rt=Δptcommand−Δptactual

Here, pt and pt+1 are the end-effector positions before and after a move; Δptcommand is the requested translation, and Δptactual is the actual translation. Both use the same fixed reference frame and length units; rt is the displacement residual for this move.

The residual captures the difference between requested motion and actual progress. Combined with gripper state, contact information, and new observations, it supports decisions to continue approaching, change orientation, or make local adjustments.

Hierarchical Memory and Long-Horizon Task State

Task memory retains the orig­i­nal goal, cur­rent stage, com­pleted mile­stones, and sup­port­ing evi­dence. Recent obser­va­tions and tool receipts remain in detailed con­text, while the full episode his­tory sup­ports on-demand retrieval. Local oper­a­tions there­fore remain con­nected to over­all task progress.

Object and region iden­ti­ties remain asso­ci­ated across inter­ac­tion rounds, while new geo­met­ric mea­sure­ments update with cur­rent obser­va­tions. The model can review ear­lier attempts and out­comes, then plan the next step using the latest posi­tions. Memory pre­serves task con­ti­nu­ity and real-time per­cep­tion updates scene con­di­tions; together, they sup­port ongo­ing deci­sions.

From Execution Feedback to Subsequent DecisionsEXECUTION RECEIPT
FeedbackHow the Model Uses It
Requested × actual displacementCom­pare motion goals with actual progress to plan fur­ther approach or pose adjust­ment.
Remaining target errorDeter­mine the direc­tion and mag­ni­tude of the next move from cur­rent error.
Gripper state × object responseAssess grasp or pulling engage­ment using test lifts, test pulls, and visual changes.
Post-action multi-view observationsUse updated object posi­tions, con­tact rela­tion­ships, and spa­tial changes in the next judg­ment.
Stage records × completion checksUpdate task mile­stones, tran­si­tion to the next stage, or finish the task.

KnowinGLOW uni­fies goal under­stand­ing, rea­son­ing, plan­ning, and EEF action gen­er­a­tion. The motion inter­face exe­cutes actions on the robot, while Har­ness man­ages tools, memory, and feed­back. The model, motion inter­face, and run­time jointly sus­tain a task loop in which every obser­va­tion and action serves the same goal.

Embodied Understanding and Task Reasoning

First-person under­stand­ing, spa­tiotem­po­ral local­iza­tion, phys­i­cal rea­son­ing, and task plan­ning sup­port teach­ing inter­pre­ta­tion and exe­cu­tion deci­sions.

GLOW’s embod­ied under­stand­ing and rea­son­ing span 18 prob­lem domains in four cat­e­gories, includ­ing object attributes and spa­tial rela­tion­ships, target-region local­iza­tion, interaction-outcome assess­ment, and task plan­ning. The exam­ples below use input images, task ques­tions, and model responses to show how these capa­bil­i­ties sup­port spe­cific deci­sions.

GEU
Comprehensive First-Person Understanding
  • Object and attribute understanding
  • Spatial relationship understanding
  • Counting and measurement estimation
  • Temporal and action understanding
  • First-person text and symbol recognition
STL
Diverse Spatiotemporal Localization
  • Object and reference-point localization
  • Free-space and region localization
  • Manipulation target-point localization
  • Temporal ordering and episode localization
PGR
Physical Grounding
  • Affordance and functional reasoning
  • State estimation and state-change reasoning
  • Trajectory and interaction-outcome reasoning
  • Contextual and multi-view reasoning
PAP
Physics-Aware Planning
  • Goal-conditioned next-action prediction
  • Multi-step manipulation planning
  • Action feasibility and success verification
  • Failure awareness and closed-loop replanning
  • Navigation and path planning

Capa­bil­ity Exam­ple · Closed-Loop Exe­cu­tion

Placing a Lid on a Teacup

External view
Wrist view
  1. 01
    Planning
    Model output · Subtask chaingrasp_knob → lift_lid → align_over_rim → lower_to_close
    Explanation

    The four steps are to grasp the wooden knob, lift the lid, move above the cup and align with its rim, and lower the lid to close it. The image shows the pre-contact phase of step 1, with the grip­per already posi­tioned on either side of the lid.

  2. 02
    Grounding
    Model output · Knob bbox [x1, y1, x2, y2][786, 136, 940, 282]
    Explanation

    Coor­di­nates refer to the 1400×700 wrist-camera image. The box center, [863, 209], is the grasp ref­er­ence point and deter­mines lat­eral offset; depth pro­vides the height dif­fer­ence. At this moment, the knob lies between the fin­gers but has not yet been grasped.

  3. 03
    Action
    Model output · delta_xyz · m · End-effector frame[0, −0.006, −0.015]
    Explanation

    A bounded pre-contact cor­rec­tion com­pen­sates for 6 mm of lat­eral offset along y and lowers the grip­per by 15 mm along z to a suit­able clos­ing height. Once the exe­cu­tion receipt returns actual dis­place­ment, the grip­per closes and per­forms a test lift to con­firm the grasp before pro­ceed­ing to step 2.

The model coor­di­nates these capa­bil­i­ties during a task: rec­og­niz­ing objects and actions, locat­ing manip­u­la­tion tar­gets, assess­ing inter­ac­tion out­comes, and plan­ning the next step. Teach­ing and real-time obser­va­tions pro­vide deci­sion con­text, while exe­cu­tion feed­back con­tin­u­ously updates task state.

03

One-Shot Teaching

One-shot teach­ing arises from the in-context learn­ing (ICL) capa­bil­i­ties of GLOW’s uni­fied mul­ti­modal autore­gres­sive model. The robot observes a single human demon­stra­tion, and Skill Demo Encoder applies task-oriented spa­tiotem­po­ral encod­ing to the con­tin­u­ous visual expe­ri­ence to pro­duce Reus­able Skill Con­text. Its seman­tic layer can align with chain-of-thought (CoT) lan­guage instruc­tions. The model com­bines this con­text with cur­rent obser­va­tions, lan­guage instruc­tions, robot state, and inter­ac­tion his­tory to gen­er­ate scene-appropriate judg­ments and actions, while its param­e­ters remain unchanged during exe­cu­tion. The water­ing exam­ples illus­trate object gen­er­al­iza­tion, while the box-storage exam­ples illus­trate gen­er­al­iza­tion across both objects and posi­tions.

From Spatiotemporal Understanding to Reusable Skill Context

Skill DIY defines a task through one con­tin­u­ous human demon­stra­tion. Skill Demo Encoder jointly inter­prets objects, phys­i­cal inter­ac­tions, and state changes across frames, con­sol­i­dat­ing task-relevant visual expe­ri­ence into Reus­able Skill Con­text. This con­text car­ries task intent and phys­i­cal inter­ac­tion knowl­edge and aligns with lan­guage instruc­tions, making the demon­stra­tion a reus­able task ref­er­ence for sub­se­quent exe­cu­tion.

Reuse pre­serves task knowl­edge while updat­ing exe­cu­tion con­di­tions: skill con­text CD pro­vides a task ref­er­ence, while cur­rent obser­va­tions, lan­guage instruc­tions, pro­pri­o­cep­tive state, and inter­ac­tion his­tory describe the con­di­tions on site. Together, they form Ct. GLOW gen­er­ates con­crete actions from these con­di­tions so that the same skill adapts to changes in tools and posi­tions. Users can also adjust goals and exe­cu­tion meth­ods through lan­guage. ICL per­forms task adap­ta­tion at run­time while model weights remain fixed.

UNIFIED AUTOREGRESSIVE IN-CONTEXT LEARNING

Single Demonstration → Reusable Skill Context → Scene-Conditioned Execution

Reusable Skill Context from a DIY DemonstrationTask-oriented spatiotemporal encoding
CD=Φθ(D)C_D=\Phi_\theta(D)
Context ConditioningSkill knowledge + current execution conditions
Ct=(Ot,L,st,Ht,CD)C_t=(O_t,L,s_t,H_t,C_D)
GLOW / ICLUnified autoregressive generation
pθ(Y∣Ct)=∏i=1Npθ(yi∣Ct,y<i)p_\theta(Y\mid C_t)=\prod_{i=1}^{N}p_\theta(y_i\mid C_t,y_{<i})
Action GenerationScene-conditioned skill execution
at∼πθ(⋅∣Ct)a_t\sim\pi_\theta(\cdot\mid C_t)

Nota­tion: D is a single demon­stra­tion; CD = Φθ(D) is the Reus­able Skill Con­text encoded from the demon­stra­tion video by Skill Demo Encoder. It car­ries trans­fer­able task intent and phys­i­cal inter­ac­tion knowl­edge, with a seman­tic layer that can align with CoT lan­guage instruc­tions.Ct com­prises cur­rent obser­va­tions Ot, lan­guage instruc­tions L, robot state st, inter­ac­tion his­tory Ht , and reus­able skill con­text CD . Fol­low­ing the nota­tion in the main text, Y = (y₁, …, yN) is the target token sequence, y<i denotes pre­vi­ously gen­er­ated tokens; pθ denotes the uni­fied sequence dis­tri­bu­tion, πθ the cor­re­spond­ing action policy, and at the cur­rent action. θbase and θtest denote deployed weights before teach­ing and weights during exe­cu­tion, respec­tively.

Skill con­text CD pro­vides a reus­able task ref­er­ence, while run­time con­text Ct updates with cur­rent obser­va­tions, lan­guage require­ments, and exe­cu­tion feed­back. ICL occurs as GLOW uses these con­di­tions to gen­er­ate judg­ments and actions token by token: task knowl­edge is reused, behav­ior adapts to the scene, and model weights remain unchanged.

One-Shot Skill DIY with GLOW

A single demon­stra­tion is encoded into reus­able skill con­text through task-oriented spa­tiotem­po­ral encod­ing. Task intent and phys­i­cal inter­ac­tion expe­ri­ence trans­fer across scenes, while GLOW gen­er­ates spe­cific actions based on cur­rent obser­va­tions and lan­guage instruc­tions. In-con­text learn­ing adapts to the task while model param­e­ters remain fixed. The image stacks, encoder, rep­re­sen­ta­tion blocks, and gen­er­al­iza­tion links are schematic illustrations of the mechanism.

One Demonstration Drives Task Adaptation

One-shot teach­ing turns GLOW’s phys­i­cal capa­bil­i­ties into task capa­bil­i­ties that users can invoke directly. A single demon­stra­tion sup­plies oper­a­tion struc­ture, cur­rent obser­va­tions estab­lish exe­cu­tion con­di­tions, and closed-loop feed­back advances com­ple­tion. The box-storage and water­ing exam­ples show adap­ta­tion to changes in objects and to joint changes in objects and posi­tions.

This inter­ac­tion lets users com­mu­ni­cate oper­a­tional intent through demon­stra­tions, while the robot adapts to the scene using its exist­ing under­stand­ing, rea­son­ing, and action capa­bil­i­ties.[8] From auton­o­mous exe­cu­tion to demonstration-driven behav­ior, GLOW pro­vides a uni­fied path for robots to receive, under­stand, and com­plete tasks.

04

GLOW’s Embodied Capabilities: From Scene Understanding to Task Completion

GLOW demon­strates strengths in both embod­ied under­stand­ing and task exe­cu­tion. Under this report’s eval­u­a­tion con­di­tions, it achieves aver­age suc­cess rates of 62.2% on RoboDojo and 86.7% on LIBERO-Pro, improv­ing on the respec­tive GPT-6 (Robocurve) and GPT-6 Astra base­lines by 40.0 and 28.5 per­cent­age points. KnowinBrain-1.5 ranks No. 1 on Embod­ied Arena’s 2D embod­ied question-answering leader­board. The fol­low­ing eval­u­a­tions and exe­cu­tion exam­ples show these capa­bil­i­ties in spa­tial judg­ment, phys­i­cal inter­ac­tion, and task com­ple­tion.

RANKED No. 1: Top-ranking evaluation results on RoboDojo, LIBERO-Pro, and Embodied Arena

RoboDojo: 62.2% Average Success Across 18 Tasks

Across 18 sim­u­lated RoboDojo tasks, GLOW achieves an aver­age suc­cess rate of 62.2%, improv­ing on GPT-6 (Robocurve)’s 22.2% by 40.0 per­cent­age points. Its aver­age score reaches 71.1, an increase of 41.3 points over the base­line’s 29.8.

Higher Success Rates and Progress Scores

Average success rate
Improvement: 40.0 percentage points
GLOW
62.2%
GPT-6 (Robocurve)
22.2%
Across 18 simulated tasks
Average score
Improvement: 41.3 points
GLOW
71.1
GPT-6 (Robocurve)
29.8
Out of 100, including progress credit for partial completion
RoboDojo · 18 simulated tasks. Success rate measures complete task execution; scores track execution progress.

LIBERO-Pro: 86.7% Average Success

Across six per­tur­ba­tion con­di­tions in LIBERO-Pro’s Object, Goal, and Spa­tial sets, GLOW run­ning inde­pen­dently achieves 86.7% aver­age suc­cess, 28.5 per­cent­age points above GPT-6 Astra’s 58.2%, and leads on five of the six con­di­tions. Improve­ments are par­tic­u­larly pro­nounced for Spa­tial posi­tion swaps and Goal task per­tur­ba­tions. Bottle-grasping and drawer-opening exe­cu­tions fur­ther show how GLOW com­bines spa­tial judg­ment, action con­trol, and real-time feed­back to com­plete tasks.

Overall Results: 28.5 Percentage Points Above GPT-6 Astra

The eval­u­a­tion covers the Object, Goal, and Spa­tial task sets, each with posi­tion swaps (Pos.) and task per­tur­ba­tions (Task). Posi­tion swaps change the ini­tial spa­tial con­fig­u­ra­tion; task per­tur­ba­tions change task con­di­tions. Suc­cess is deter­mined by the sim­u­la­tor’s native cri­te­ria.

LIBERO-Pro Success Rates by Method

Success rate (%) · Descending average
Single-policyMixture-of-policies
CategoryMethodSpatialObjectGoalAverage ↓
TaskPos.TaskPos.TaskPos.
Mixture-of-policiesRPent / GPT-6 Astra Codex1009810099889997.3
Mixture-of-policiesHarness VLA (Claude Code)94808890878787.7
Single-policyGLOW59979797977386.7
Mixture-of-policiesHarness VLA (Codex) / RPent GPT-5.5 · xhigh81699491756679.3
Mixture-of-policiesRPent Flash Mode Molmo2-8B · visual localization79708693746577.8
Mixture-of-policiesRPent / Qwen3.6 27B no-reasoning82788384686877.2
Single-policyASPIRE60519598458171.7
Single-policyGPT-6 Astra41248959419558.2
Single-policyπ_RLinf42597178454256.2
Single-policyRATS31296361364343.8
Single-policyCap-X14121822172618.2
Single-policyπ₀.₅12011723813.2
Single-policyAtomVLA1160101126.7
Single-policyX-VLA0082913.3
Single-policyMolmoAct0006001.0
Single-policyπ₀0002000.3
Single-policyOpenVLA0000000.0
Single-policyNORA0000000.0
Pos. denotes position swaps; Task denotes task perturbations. All six conditions use a 0–100% scale, with values shown as integers. Averages are displayed to one decimal place.

With a single policy, GLOW achieves an aver­age suc­cess rate of 86.7% across the six LIBERO-Pro tests, rank­ing first among the single-policy models in the table and exceed­ing runner-up ASPIRE by 15.0 per­cent­age points. Com­pared with mixture-of-policies sys­tems, GLOW approaches Har­ness VLA (Claude Code) at 87.7% and exceeds the other three such sys­tems listed, demon­strat­ing cross-task exe­cu­tion with a uni­fied policy.

These results show that a single GLOW policy can per­form com­pa­ra­bly to some multi-policy sys­tems while main­tain­ing the high­est aver­age among the single-policy models eval­u­ated.

Execution Examples: Spatial Judgment and Action Feedback

The six paired record­ings below show both sys­tems exe­cut­ing the same tasks from iden­ti­cal ini­tial states. GPT-6 Astra with native RGB input is on the left, and GLOW is on the right. GLOW passes the native suc­cess check in these exam­ples; the base­line does not. Fail­ing the check, how­ever, does not mean every part of the oper­a­tion was incor­rect. In par­tic­u­lar, Astra com­pletes the main manip­u­la­tion steps in all three bowl-to-plate tasks, with dif­fer­ences con­cen­trated in final pre­cise place­ment. The videos and actual execution-step counts help dis­tin­guish ben­e­fits from smoother motion, more accu­rate place­ment, and addi­tional cor­rec­tive actions.

Wine bottle into bowlGPT-6 Astra · Native check failed863 native sim­u­la­tion steps
Wine bottle into bowlGLOW · Native check passed157 native sim­u­la­tion steps

Select a task above to view paired record­ings: Astra on the left, GLOW on the right. Within each video, the exter­nal view is on the left and the wrist view on the right. Play­back defaults to 0.5×, and each video can be viewed fullscreen. Orig­i­nal exe­cu­tion dura­tions are pre­served; task stages are not forced into syn­chro­niza­tion.

Precise Placement Determines Task Completion

In all three bowl-to-plate record­ings, Astra selects the cor­rect target and com­pletes grasp­ing and trans­port, but its final place­ment does not meet the native suc­cess cri­te­ria. The dif­fer­ences are con­cen­trated in final align­ment, rather than errors through­out the entire task.

GLOW adjusts in stages based on the object’s posi­tion rel­a­tive to the target plate, leav­ing oppor­tu­ni­ties to observe and cor­rect before con­tact. For exam­ple, the bowl beside the cookie box is low­ered in 10, 8, and 5 mm incre­ments and ulti­mately passes the check. In the bottle task, Astra’s final hor­i­zon­tal dis­tance between the bottle and bowl ref­er­ence points is approx­i­mately 3.4 cm, beyond the task’s 3 cm thresh­old; GLOW’s is approx­i­mately 6.9 mm. This thresh­old applies only to this bottle task.

Smoother Actions, but Not Faster in Every Case

In the bottle task, Astra exe­cutes 863 native sim­u­la­tion steps, com­pared with GLOW’s 157. Staged approach and test lifts to con­firm the grasp reduce empty grasps and repeated adjust­ments, making trans­port and place­ment more con­tin­u­ous. The center-bowl task also decreases from 330 to 254 steps.

In other exam­ples, the ben­e­fit mainly comes from cor­rec­tion. Step counts are nearly iden­ti­cal for the bowl beside the cookie box, but GLOW com­pletes pre­cise place­ment. The bowl beside the ramekin and the can-into-basket task require more steps for obstruc­tion han­dling and target cor­rec­tion, respec­tively. GLOW’s strengths there­fore include both reduc­ing unpro­duc­tive attempts and using addi­tional actions to advance com­ple­tion.

These counts are native sim­u­la­tion steps: 20 steps equal 1 second of sim­u­la­tion time, exclud­ing model wait­ing time. Because the sys­tems have dif­fer­ent stop­ping con­di­tions, these counts do not sup­port a strict speed com­par­i­son between suc­cess­ful poli­cies.

Feedback Enables Ongoing Correction

While open­ing a drawer, GLOW requests a move of 24.3 mm toward the cab­i­net and 20.3 mm down­ward, but achieves only 1.6 mm and 0.7 mm. The exe­cu­tion receipt reveals the obstruc­tion, prompt­ing pose adjust­ment, staged approach, and pulling of the target drawer.

In the can-into-basket task, both sys­tems ini­tially select the wrong object. Astra stops after trans­port. GLOW receives fail­ure feed­back, rechecks the scene, grasps the cor­rect can, and com­pletes the task. This illus­trates feedback-based cor­rec­tion rather than more accu­rate ini­tial recog­ni­tion.

These exam­ples show closed-loop ben­e­fits in grasp con­fir­ma­tion, obstruc­tion han­dling, and pre­cise place­ment. They are obser­va­tions of com­plete sys­tems, not abla­tion find­ings for indi­vid­ual tools. In some record­ings, the suc­cess check trig­gers while the object is still held by the grip­per, so long-term place­ment sta­bil­ity is not estab­lished.

Embodied Arena: KnowinBrain-1.5 Ranks No. 1 in Embodied QA

On Embod­ied Arena’s 2D-Embodied QA Bench­marks leader­board, KnowinBrain-1.5 ranks first among 20 embod­ied models with a Total Score of 65.62. This score is cal­cu­lated from nine bench­mark scores. The table below lists the top ten models in descend­ing order of total score.

EMBODIED ARENA / KNOWINBRAIN-1.5

65.62 Across Nine Benchmarks · No. 1 in Embodied QA

2D Embodied QA · 20 models
1st
Total Score
65.62
Evaluation coverage
9 benchmarks
Question Answering · 2D-Embodied QA Benchmarks

Overall Leaderboard: Top Ten

RankModelOverall score
Total Score ↓
1KnowinBrain-1.565.62
2ABot-ER-Preview65.14
3Wall-Brain-Preview64.03
4Embodied-R1.563.33
5KnowinBrain61.31
6RynnBrain-8B55.53
7Mimo-Embodied-7B55.29
8IflyBot-VLM-8B52.10
9PelicanVL-7B49.62
10EmbodiedBrain-7B48.10

The leader­board is sorted by Total Score in descend­ing order and covers 20 embod­ied models.

KnowinBrain-1.5: Results on Nine Benchmarks

BenchmarkScore
UniEQA45.62
OpenEQA66.95
VSI62.83
ERQA47.89
RoboVQA71.88
Where2Place75.48
EmbSpatial83.76
Cosmos66.47
VABench-Point69.67

Over­all score: 590.55 ÷ 9 = 65.6167, dis­played as 65.62 to two dec­i­mal places. Sources: Embodied Arena official task leaderboard · KnowinBrain-1.5 model page · Updated: 2026-09-21.

Capability examples · Swipe to browse
01 / 09
05

Outlook: Bringing Integrated Embodied Capabilities Into the Physical World

General-purpose models expand knowl­edge, under­stand­ing, and rea­son­ing, while spe­cial­ized embod­ied models center on phys­i­cal inter­ac­tion to build inte­grated capa­bil­i­ties from scene under­stand­ing to auton­o­mous action. Knowin con­tin­ues to pursue this approach: train­ing models on phys­i­cal expe­ri­ence at scale, coor­di­nat­ing cog­ni­tion and action through a uni­fied archi­tec­ture, and adapt­ing to envi­ron­men­tal changes through closed-loop exe­cu­tion.

Gen­eral and spe­cial­ized capa­bil­i­ties rein­force each other. Just as lan­guage models under­stand cre­ative intent and video models learn tem­po­rally con­tin­u­ous visual gen­er­a­tion, embod­ied models sys­tem­at­i­cally learn con­tact, motion, state changes, and action con­se­quences. Spe­cial­ized data, train­ing objec­tives, and model archi­tec­tures allow these capa­bil­i­ties to deepen within their respec­tive tasks.

Phys­i­cal inter­ac­tion pro­vides direct evi­dence for learn­ing and deci­sions: an object moving with the hand indi­cates grasp state; approach and align­ment out­comes reveal spa­tial rela­tion­ships; and con­tin­ued exe­cu­tion after envi­ron­men­tal changes reflects task plan­ning. GLOW brings objects, space, actions, and out­comes into the same learn­ing and exe­cu­tion process, devel­op­ing rea­son­ing and decision-making capa­bil­i­ties for the phys­i­cal world.

High-quality phys­i­cal expe­ri­ence at scale is the foun­da­tion of this system. Real and syn­thetic expe­ri­ence spans action con­di­tions, con­tact pro­cesses, and state changes, enabling the model to learn manip­u­la­tion rela­tion­ships across objects and scenes and adjust actions from new obser­va­tions. Expe­ri­ence gen­er­a­tion, model train­ing, and exe­cu­tion feed­back jointly build phys­i­cal capa­bil­i­ties.

KnowinDream and KnowinWorld expand phys­i­cal expe­ri­ence. KnowinGLOW uni­fies under­stand­ing, rea­son­ing, plan­ning, and action gen­er­a­tion, while KnowinAgent man­ages task memory, replan­ning, and exe­cu­tion feed­back. Together, they form a system for closed-loop exe­cu­tion and capa­bil­ity accu­mu­la­tion. LIBERO-Pro and RoboDojo results demon­strate its exe­cu­tion per­for­mance, while real-robot exam­ples using a single demon­stra­tion fur­ther illus­trate task adap­ta­tion.

GLOW pro­vides inte­grated under­stand­ing, rea­son­ing, plan­ning, and exe­cu­tion capa­bil­i­ties and can also col­lab­o­rate with general-purpose models such as GPT. Auton­o­mous capa­bil­ity and open col­lab­o­ra­tion serve a shared goal: enabling robots to under­stand tasks, orga­nize actions, and adapt to new con­di­tions through teach­ing and feed­back.

Knowin will con­tinue advanc­ing GLOW’s phys­i­cal learn­ing and gen­er­al­iza­tion capa­bil­i­ties, enabling robots to act autonomously in chang­ing envi­ron­ments and under­stand and com­plete new tasks after a single teach­ing exam­ple.

06

References

[1]Google DeepMind. Gemini Robot­ics 1.5: Push­ing the Fron­tier of Gen­er­al­ist Robots with Advanced Embod­ied Rea­son­ing, Think­ing, and Motion Trans­fer. 2025. [Technical Report]

[2]Phys­i­cal Intel­li­gence. π₀.₅: A VLA with Open-World Gen­er­al­iza­tion. 2025. [Technical Report]

[3]Figure AI. Helix 2.5: Zero-Shot 30-Home Gen­er­al­iza­tion. 2026. [Technical Report]

[4]NVIDIA. GR00T N1: An Open Foun­da­tion Model for Gen­er­al­ist Humanoid Robots. 2025. [Technical Report]

[5]ByteDance Seed Robot­ics. GR-3 Tech­ni­cal Report. 2025. [Technical Report]

[6]Skild AI. Intro­duc­ing S1: In-Context Learn­ing for Robot­ics. 2026. [Technical Report]

[7]Phys­i­cal Intel­li­gence. π₀.₇: A Steer­able Model with Emer­gent Capa­bil­i­ties. 2026. [Technical Report]

[8]Gen­er­al­ist AI. GEN-1.5: Embod­ied Foun­da­tion Models are One-Shot Learn­ers. 2026. [Technical Report]

Citation

Please cite this article as:

KNOWIN. GLOW: A Generative Learning Framework for General-Purpose Embodied Intelligence.

https://knowinai.com/tech.html 
@article{knowin2026glow,
  author = {{KNOWIN}},
  title  = {{GLOW}: A Generative Learning Framework for General-Purpose Embodied Intelligence},
  year   = {2026},
  month  = {September},
  url    = {https://knowinai.com/tech.html}
}
Enlarged technical diagram