EgoSwarm

Learning Decentralized Robot Swarm Intelligence from Human Swarm Demonstrations

Xitong Chen 1,† , Ziqiang Liu 1,† , Min Li 1,* , Yiqun Li 1 , Zhuoping Yin 1
1 State Key Laboratory of Intelligent Manufacturing Equipment and Technology,
Huazhong University of Science and Technology, Wuhan, China.

† Equal contribution   * Corresponding author
EgoSwarm framework overview
50+ hours Human collaboration
30 Scenario categories
72.2 % Mean task success
30 Hz Closed-loop control

Abstract

Advancing from individual embodied intelligence to swarm embodied intelligence is a key step toward scalable, general-purpose embodied intelligence. Complex tasks often require multiple agents to divide responsibilities, coordinate handovers, and manipulate shared objects collaboratively. However, existing robot learning methods rely primarily on individual demonstrations, which provide limited supervision for modeling dependencies among agent actions, while collecting multi-robot demonstrations at scale remains difficult. Human egocentric demonstrations offer a scalable alternative, yet distributed observations from multiple participants cannot directly supervise decentralized policies. We introduce EgoSwarm, the first robot learning framework for learning decentralized multi-robot collaboration from human swarm demonstrations. EgoSwarm collects more than 50 hours of collaborative experience using a robot-free multi-person egocentric system. We introduce the Swarm Interaction Packet (SIP) and Swarm Interaction Token (SIT) to recover cross-agent interaction relations from distributed observations and transform this experience into local supervision consistent with deployment-time information constraints. We further propose SwarmVLA, which transfers knowledge of swarm behavior to decentralized policies conditioned on local observations, local history, and limited neighbor messages. SwarmVLA achieves a mean success rate of 72.2% across three categories of multi-robot collaborative tasks.

System Design

EgoSwarm Capture synchronously records multi-person egocentric demonstrations without robots during data collection. Each sensing head integrates six cameras, an IMU, and UWB sensing. The robot platform reuses the same sensing interface and independently performs perception, local interaction processing, and SwarmVLA inference on NVIDIA AGX Thor.

Human demonstration setup and shared robot sensing platform

Training Data

The training data consists of more than 50 hours of human collaborative experience across 30 scenario categories, robot adaptation data for embodiment alignment, and joint robot teleoperation data for task-specific action supervision.

Training data pyramid

SwarmVLA

Multimodal egocentric recordings are temporally aligned and reconstructed into Swarm Interaction Packets (SIPs). A causal teacher learns from collective interaction supervision and transfers coordination representations to a decentralized student policy. During deployment, each robot constructs local Swarm Interaction Tokens (SITs) and generates actions using local observations, history, and quality-aware neighbor messages.

SwarmVLA interaction reconstruction, causal teacher, and decentralized policy architecture

Autonomous Policy Rollouts

Deployment experiments are conducted across three representative tasks to evaluate decentralized multi-robot collaboration.

Multi-Robot Flexible Handover (MRFH)

Three robots sort laundry through coordinated handovers and basket loading, testing conditional handovers and temporal coordination.

80.0% success rate

Dynamic Collaborative Transport (DCT)

Two robots load a box onto a cart before a third takes over transport, testing joint support and role transitions.

73.3% success rate

Large-Scale Deformable Object Manipulation (LSDOM)

Multiple robots jointly manipulate a wire harness, testing sustained coordination through a shared deformable object.

63.3% success rate

Experimental Results

EgoSwarm achieves a mean success rate of 72.2% across three tasks, exceeding Dec-VLA-Comm by 22.2 percentage points. Each method is evaluated in 30 independent trials per task.

Task success (%) · 30 independent trials per task and method
Method MRFH DCT LSDOM Mean
Local-Rule 20.0 26.7 10.0 18.9
Independent VLA 43.3 46.7 36.7 42.2
Dec-VLA-Comm 63.3 56.7 30.0 50.0
Centralized Joint VLA* 80.0 50.0 46.7 58.9
EgoSwarm (ours) 80.0 73.3 63.3 72.2

* Global-observation reference. Evaluation primarily uses three robots.

Ablation Studies

Robot-only 22.2%
50% human swarm data 53.3%
Without SIT 38.9%
Without quality gate 45.6%
Full SwarmVLA 72.2%

Communication Robustness and Runtime

66.7 %

Mean success under 30% packet loss and 100 ms additional delay .

P95 end-to-end latency
38.6 ms
Neighbor traffic per robot
11.8 KB/s
Relative translation RMSE
4.8 mm

BibTeX

@misc{chen2026egoswarm,
  title={EgoSwarm: Learning Decentralized Robot Swarm Intelligence
         from Human Swarm Demonstrations},
  author={Xitong Chen and Ziqiang Liu and Min Li and Yiqun Li and Zhuoping Yin},
  year={2026},
  url={https://xitong-c.github.io/EgoSwarm_Page/}
}