Alibaba Qwen Introduces QwenGyre: Elastic Reinforcement Learning RL for XLong-Horizon Agents

Alibaba’s Qwen team has introduced QwenGyre, an elastic reinforcement learning RL framework for extreme-long-horizon agent training. QwenGyre is an online reinforcement learning system from Alibaba that trains coding agents through unmodified, black-box harnesses. It targets rollouts that run for hours and process close to 1M tokens. The QwenGyre elastic RL framework was used on Qwen’s flagship Qwen 3.8 2.4T model. It rests on 2 core ideas. It moves idle GPUs between rollout and training. It also turns branching agent histories into bounded training samples.

TL;DR

• Size: A training framework, not a model. Tested on Qwen 3.6 122B and Qwen 3.8 2.4T (A95B per Qwen), with ~700K tokens per rollout.
• Runs on: Multi-node GPU clusters. The 122B runs used 24 to 32 nodes; the 2.4T run used 192 nodes (4 cells of 48). GPU model not disclosed. Megatron trains, SGLang serves rollouts.
• Performance: 1.38x to 1.78x faster than Async, 1.21x to 1.85x faster than Colocate, at matched scores.
• Best: 1.85x end-to-end speedup over Colocate on TerminalBench.
• Worst: Only 1.21x over Colocate on the 2.4T model, where many queries ran near the timeout.
• Bottom line:
  • Best: NL2RepoBench rises from 52.5% to 58.5% in 48 steps.
  • Worst: the paper links no public code release.

What problem does QwenGyre solve?

The research team defines ‘XLong-horizon’ tasks as agent runs spanning hours and hundreds of model and tool interactions. Examples include generating full repositories from specs, as in NL2Repo-Bench.

Two problems appear at this scale:

• First, rollouts have long tails. In a Colocate setup, training waits for the slowest execution while GPUs idle. In an Async setup, fixed GPU pools cannot lend spare capacity to each other.
• Second, harnesses like Claude Code compact context, spawn sub-agents and retry. One execution becomes a branching tree, not a single trajectory. Expanding every path duplicates shared prefixes and inflates training cost.

How does the Elastic Scheduler work?

QwenGyre splits GPUs into cells that share a training layout. Each cell can serve rollouts or host a full training replica. A core cell owns the authoritative weights and optimizer state. Satellite cells join training later.

The scheduler tracks a rollout waterlevel: queued and running executions. As executions finish, cells switch to training once the remaining cells can still serve all live work. Model calls are rerouted through a proxy, so the harness keeps running.

Key mechanics:

• KV-cache blocks migrate over RDMA during rerouting, reducing prefix recomputation.
• Satellites pull core weights through Mooncake’s Transfer Engine and join an ongoing batch.
• Streaming training lets cells claim micro-steps as data becomes ready.
• Average role switches took 8.52 s (rollout to training) and 3.46 s (training to rollout).

How does the Trajectory Processor build training samples?

A proxy records exact token IDs and behavior log-probabilities using token-in, token-out (TITO). Calls are stored as prefix-shared trajectory trees.

• Partial scoring: Timed-out executions are still scored on their preserved workspace. Evaluator failures are excluded, never treated as a zero reward.
• Bounded sampling: At most 5 trajectories per execution are kept, ranked main > main-summary > sub-agent > sub-agent-summary.
• Deduplication: Shared targets count once, and only policy-generated tokens are trained.
• Fair weighting: Losses average within each execution, so extra branches do not inflate its weight.

A cap of 5 matched uncapped training scores while cutting forward-backward time by 25.2%. Training uses an adapted GSPO with 16 rollouts per group.

How fast is QwenGyre? Benchmark results

All runs used Claude Code 2.1.220 as the harness, with equal GPU budgets and matched staleness.

Workload (model)	vs Async	vs Colocate
NL2RepoBench (Qwen 3.6 122B, 48 steps)	1.42x to 1.53x	1.36x to 1.47x
DeepSWE (Qwen 3.6 122B, 24 steps)	1.569x to 1.572x	1.610x to 1.824x
TerminalBench (Qwen 3.6 122B, 24 steps)	1.383x to 1.430x	1.809x to 1.849x
NL2RepoBench (Qwen 3.8 2.4T, 48 steps)	1.78x	1.21x

On the 2.4T model, QwenGyre finished 48 steps in 75.42 hours. Async took 134.55 hours and Colocate took 91.47 hours. Evaluation pass rate rose from 52.48% to 58.54%.

How does QwenGyre compare with other agentic RL frameworks?

Feature	QwenGyre	veRL	AReaL	Agent Lightning
Developer	Alibaba Qwen	ByteDance Seed + community	Tsinghua IIIS + Ant Group	Microsoft
License	Not disclosed	Apache-2.0	Apache-2.0	MIT
GPU placement	Elastic cells, switch mid-rollout	Flexible device mapping, HybridEngine resharding	Fully asynchronous	Not disclosed
Black-box harness	Yes, via proxy	Not disclosed	Yes, via base_url swap	Yes, near-zero code change
Training / rollout	Megatron / SGLang	FSDP, Megatron / vLLM, SGLang	Megatron, FSDP, Archon / vLLM, SGLang	Not disclosed
Largest stated scale	Qwen 3.8 2.4T	671B models	Not disclosed	Not disclosed
Headline speedup	Up to 1.85x vs Colocate	Not disclosed	2.77x vs synchronous (v0.3)	Not disclosed
Public code	Not disclosed	Yes	Yes	Yes

Speedups come from each project’s own setup and are not directly comparable.

Key Takeaways

• QwenGyre moves GPUs between rollout and training without stopping live agents.
• Trajectory trees cap samples at 5 per execution and train shared prefixes once.
• Timed-out runs still get scored on their partial work.
• Up to 1.85x faster than Colocate and 1.78x faster than Async.
• Qwen 3.8 2.4T NL2RepoBench improved from 52.5% to 58.5% in 48 steps.

---

Check out the full QwenGyre paper on arXiv and Qwen’s announcement post. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.