PhysisServe: Efficient Autoregressive Denoising Serving with Fine-Grained Stage-wise Pipelining
Introduction
The next destination for artificial intelligence is the physical world. A physical agent acts in a world that never stops changing. A slower LLM usually means a longer wait for the same answer. For a physical agent, a delay can make an action obsolete before it is executed. A physical agent therefore places two requirements on its model: it must be large enough to understand and predict the physical world accurately, and it must output quickly enough to keep pace with physical processes and actuators. In reality, the largest frontier robotics models are still much smaller than LLMs: general-purpose robot models remain at the scale of several billion parameters, while open-source LLMs have entered the trillion-parameter range [1]—the gap reflects a hard constraint: the model must fit on hardware that can run it in real time, so intelligence is limited by latency and cost. Physical AI faces a fundamental serving challenge: how to run larger, more capable models at the output rates physical tasks require. This is the problem PhysisServe addresses.
The community has not yet reached a fully unified definition or boundary for “world models.” Rather than comparing the merits of different architectures, we discuss two implementation types directly relevant to the serving system in this work, starting from the input and output forms in the interaction process.
-
Action-Conditioned World Model (ACWM). This type of model takes the current observation or frame of the environment, together with the agent’s action or instruction, as conditions and generates subsequent observations or frames. Actions or instructions can continue to enter during generation, so the generated trajectory changes in real time with the agent’s behavior and can serve as an environment for agent training or evaluation. Dreamer 4 and Genie 3 both place the environment state and the agent’s action or instruction in the generation loop and continuously generate future observations as the interaction proceeds [2, 3].
-
World Action Model (WAM). This type of model takes the robot’s observation of the environment as input and continuously generates subsequent actions. Its output is no longer a future frame but an action sequence that can be used directly to control the robot, so it can serve as a policy or policy module. DreamZero and LingBot-VA 2.0 both read the robot’s continuous observations and convert the model output into the next executable action [4, 5].
Both ACWM and WAM can generate through autoregressive denoising. Autoregressive denoising and token-by-token generation in large language models are both autoregressive; the difference is whether a later output must wait for the previous one to finish. An LLM generates one token at a time, and the next token cannot begin computation until the previous token has passed through all Transformer blocks, so tokens are generated in sequence (left of Figure 1). Autoregressive denoising instead generates consecutive frames in units of chunks: each chunk starts from pure noise and gradually becomes clear through several denoising steps. A later chunk does not need to wait for the previous chunk to finish denoising; whenever the previous chunk completes one denoising step, the later chunk can read the still-noisy KV cache it left at that step and execute its corresponding denoising step [6]. The denoising of adjacent chunks can therefore be staggered and overlapped, allowing chunks to complete continuously at a short interval (right of Figure 1). In this post, output latency refers to this steady-state interval between completed chunks, not the response time to a new input.
The output frequency required by a model depends on the task’s requirements for response speed and action-control precision. In humanoid-robot simulation training, the simulation frequency can reach several thousand Hz in order to accurately model contact between the robot and the environment [7]. In deployment, control commands generally need to be updated more frequently closer to the low-level actuators: the semantic-planning layer updates more slowly, typically below 50 Hz; the action and servo layers need more frequent adjustment and can run at several hundred hertz [1].
However, a large parameter count and a high output frequency are difficult to achieve simultaneously. Larger models can represent the world more accurately, but each output requires more computation. Autoregressive dependencies limit how that work can be parallelized. This creates a trade-off: model capability calls for scale, while real-time execution favors smaller models. This trade-off is also reflected in publicly available models. For ACWM, LingBot-World uses a MoE architecture with approximately 28B parameters (approximately 14B active per step) and outputs at 16 FPS; MAGI-1 has 24B parameters and generates a 24-frame chunk in about 0.98 seconds on 24 high-end GPUs, corresponding to approximately 24 FPS [8, 9]. For WAM, DreamZero has 14B parameters and generates an action chunk at approximately 7 Hz; LingBot-VA 2.0 has approximately 15.3B training parameters (approximately 2.5B active per token), with a generation latency of 142 ms/chunk, corresponding to approximately 7.0 chunks/s and a peak asynchronous control frequency of 225 Hz [4, 5]. Figure S0 outputs joint-level commands at 1 kHz with 10M parameters [10]. If a large-parameter model can also maintain a sufficiently high output frequency, its applications can expand from video generation and high-level interaction to high-frequency simulation and low-level control; the output frequencies required by these settings still differ from those provided by current large-parameter models by approximately two orders of magnitude.
No autoregressive-denoising inference engine has yet reconciled model scale and output frequency (Figure 2). Wave Forcing partitions at the denoising-step granularity: each GPU holds the complete model and handles one denoising step, and chunks and denoising steps advance diagonally [11]. A stream can use at most as many GPUs as there are denoising steps, and its output latency cannot be shorter than one complete-model forward pass—larger models make each forward pass slower, so the output frequency decreases. StreamDiffusionV2 partitions at the block granularity: the model is split across multiple GPUs as a pipeline, and computations for the same block at different denoising steps are combined into one batch to improve utilization [6]. This creates two limitations: a stream can use at most as many GPUs as there are blocks, so the GPU count is capped by the number of blocks; and as the number of denoising steps increases, more computation is combined into the batch, making each forward pass slower, while batch composition and scheduling remain inflexible. Once the number of GPUs exceeds this cap, the output frequency no longer increases; larger models achieve a lower output frequency.
We introduce PhysisServe, an inference framework for autoregressive denoising that brings large models and high-frequency output together. With sufficient compute, PhysisServe is designed to sustain the output rate a task requires as models scale to tens or hundreds of billions of parameters—or even trillions. PhysisServe partitions the model into multiple stages, where each stage corresponds to one Transformer block at one denoising step, and uses the stage as the basic unit of compute allocation and scheduling. The system is organized around three levels: speeding up one stream, sharing compute, and scaling to multiple streams:
-
Stage-wise pipeline. The denoising-step and Transformer-block dimensions are pipelined simultaneously, expanding the number of GPUs that one stream can use to the product of the two dimensions. In steady state, the output latency is determined by the computation time of the slowest stage; increasing the number of denoising steps or model depth mainly increases the pipeline depth and the required GPU count, rather than directly lowering the output frequency.
-
Time-sharing. Stages are decoupled from physical GPUs, so one GPU can execute multiple stage tasks in alternating time slices. Tasks from different chunks and denoising steps on the same block are combined into a batch by the scheduler and executed together, reusing weight loading and kernel-launch overhead. GPU idle time can therefore be occupied by other chunks, so each stage no longer needs to occupy a GPU exclusively; when the lowest possible output latency is not required, a single stream can use fewer GPUs while still meeting its target SLO.
-
Dynamic placement of virtual ranks. The computation tasks and runtime state of each stream at its stages are packaged into virtual ranks, which are then mapped to physical GPUs. When a new stream arrives, the scheduler chooses placements according to the remaining capacity on each GPU; it admits the stream only when all stages of the stream can be accommodated, and running streams are not migrated. This preserves the stability of existing services, balances load across GPUs, and makes use of fragmented free capacity.
We evaluate PhysisServe on five backbones commonly used for ACWM and WAM: Wan2.1 1.3B, Wan2.2 TI2V 5B, Wan2.1 14B, LTX-2.5 22B, and MiniMax H3 33B [12]aModel configuration: 30/30/40/48/50 blocks; 4 denoising steps; only the video branch is counted for LTX-2.5. Wan2.1 uses StreamDiffusionV2 weights [6]; the remaining models use noisy-KV fine-tuning, built on Rolling Forcing [13]. Historical window: 18 latents for MiniMax H3 and Wan2.2 TI2V 5B; 21 for Wan2.1 1.3B and LTX-2.5; 6 for Wan2.1 14B, with one additional attention sink., with weights neither pruned, compressed, nor quantizedbEach chunk contains 1 latent. Wan2.1, Wan2.2 TI2V, and MiniMax H3 decode to 4 frames; LTX-2.5 decodes to 8 frames. Timing covers only DiT denoising and excludes VAE decoding.. When each stage occupies one GPU, the measured single-stream output latency of the 33B MiniMax H3 is 4.1 ms. Figure 3 shows the output frequency of different models at different GPU counts; the raw measurement for MiniMax H3 with one GPU per stage is 4.083 ms, corresponding to approximately 245 chunks per secondc1000 / 4.083 = 244.9/s, rounded to 245., equivalent to approximately 1000 frames per second under an ACWM output conventiondACWM convention: 1 latent decodes to 4 frames; 244.9 × 4 ≈ 980 frames/s., equivalent to approximately 3900 control steps per second under a WAM output conventioneLingBot-VA 2.0 convention: 16 control steps per latent; 244.9 × 16 ≈ 3919 Hz., or 120 action chunks per secondfDreamZero convention: 2 latents per action chunk; 244.9 / 2 ≈ 122.5 action chunks/s.. These rates are conversions of measured DiT throughput, not measured closed-loop control rates. The minimum output latency across the five models is 1.9–7.6 ms, determined by block computation and inter-GPU transfer. It does not increase monotonically with parameter count in these tests: the 33B MiniMax H3 achieves a shorter interval than the 1.3B Wan2.1.
Stage-wise pipeline
PhysisServe decomposes autoregressive denoising into stage-level tasks that can overlap across chunks, allowing different chunks to occupy different pipeline stages simultaneously and reducing single-stream output latency to the computation time of one stage when sufficient compute is available.
A chunk passes through all stages in sequence, and the computation on one stage is a task. Each task depends on the output of the same chunk at the preceding stage and on the KV cache produced by the preceding chunk at the same stage. The noisy KV cache means that a later chunk does not need to wait for the previous chunk to finish denoising: it can read the KV cache left by the previous chunk at the current denoising step and execute out of step with that chunk across different stages. The tasks of different chunks therefore advance along diagonals and form a pipelined overlap (Figure 4).
Consider a model with two denoising steps and four blocks. PhysisServe divides the model into eight stages and maps them to eight GPUs (Figure 5). Chunks advance through the pipeline stage by stage; once the pipeline is full, one chunk completes per pipeline interval. Model depth and the number of denoising steps mainly increase the pipeline depth and the required GPU count, rather than the output latency in steady state. To keep each stage focused on computation, the KV cache remains on its local GPU, and only the hidden state is transferred between GPUs; the downstream GPU buffers the received hidden state until the next pipeline interval, allowing transfer to overlap with computation. There is no global synchronization between GPUs.
Figure 6 measures the single-stream output latency of five models at different GPU counts. When there are few GPUs, adding GPUs reduces output latency; once each stage occupies one GPU, the latency converges to 1.9–7.6 ms. The convergence value is determined jointly by the computation of one stage and transfer between adjacent stages, while differences between models mainly come from block computation, chunk-token count, history length, and communication overhead.
The Stage-wise pipeline establishes the lower bound for single-stream output latency, but this configuration requires one GPU per stage. For scenarios that do not require the lowest possible latency, idle resources in the pipeline can be reused to reduce the number of GPUs required by one stream.
Single-stream time-sharing
When the target output latency is less stringent, PhysisServe uses time-sharing and batching to let multiple stages share one GPU, reducing the number of GPUs required by one stream while meeting its SLO.
The computation times of different stages in a Stage-wise pipeline are not identical, so a fixed one-GPU-per-stage mapping leaves idle time. Time-sharing lets one GPU carry multiple stages in different time slices and recovers these gaps. Ready tasks from different chunks and denoising steps on the same block share model weights and can be batched when they have no mutual dependency; batching reuses weight-loading and kernel-launch overhead and reduces total task time. Together, the two mechanisms allow one GPU to complete more tasks within one output latency.
As shown in Figure 7, consider an example with two denoising steps and four blocks. The baseline uses a static assignment of stages across GPUs without batching. Because stage computation times differ, the GPUs can be imbalanced, with some idle while waiting for other stages, so three GPUs are needed to reach the target output frequency. PhysisServe instead uses time-sharing to let multiple stages share a GPU and combines tasks from different chunks on the same block into a batch. Batching increases the execution time of an individual batch, but amortized over its multiple tasks it reduces fixed costs such as weight loading and kernel scheduling, lowering the average time per task. Together with stage-level time-sharing, PhysisServe needs only two GPUs to reach the same output frequency.
The comparison between the execution time of one batch and the linear reference is shown in Figure 8. The measured time is below the linear boundary formed by executing samples one by one, showing that although batching increases the execution time of one batch, it amortizes fixed costs such as weight loading and kernel scheduling.
As shown in Figure 9, we evaluate the single-stream SLO corresponding to two scheduling strategies under different GPU configurations: a conventional pipeline with Stage-wise static partitioning and no batching, and a Stage-level time-sharing scheme. Here, SLO is represented by the measured chunk output latency. Compared with the former, Stage-level time-sharing (dark) lowers the required SLO in most configurations. For example, on one GPU with Wan2.2 5B, PhysisServe reduces the single-stream SLO from 173 ms to 98 ms, a reduction of approximately 43%; on two GPUs with Wan2.1 1.3B, it reduces the single-stream SLO from 187 ms to 139 ms, a reduction of approximately 26%. Overall, PhysisServe performs better than the conventional pipeline in most configurations, with a particularly clear advantage when GPU resources are constrained. This shows that PhysisServe’s Stage-level time-sharing orchestration can improve compute allocation and increase output frequency for a fixed model and compute budget.
Single-stream time-sharing extends the Stage-wise pipeline from a fixed one-GPU-per-stage mapping to multiple stages sharing GPUs, allowing the system to meet a single-stream SLO with fewer resources. For ACWM, practical deployment typically requires continuous interactive prediction for multiple users or agents at once, so this resource reuse must be extended further across multiple requests.
Multi-stream joint scheduling
When ACWM expands from serving one user or agent to serving multiple users or agents, PhysisServe lets requests share the entire GPU pool and performs time-sharing and batching across requests, maximizing the number of streams served simultaneously under finite compute and per-stream SLO requirements.
Equal partitioning into GPU groups causes the GPU count to be rounded up, and each batch contains only one stream’s tasks, so idle memory and compute time cannot be shared across requests. Joint scheduling lets multiple requests share the whole GPU pool: GPUs take turns executing different requests at batch granularity, and tasks from different requests on the same block can be batched across streams (Figure 10).
PhysisServe divides the stages of one stream into several groups, each called a virtual rank: it handles a group of stages and stores their corresponding KV cache, while tasks at different denoising steps on the same block can be batched within it. A virtual rank is the basic placement unit, and one GPU can carry virtual ranks from multiple streams simultaneously. New requests are placed according to memory, compute time, and SLO, using Best-Fit [14] together with Limited Discrepancy Search (LDS) [15]: virtual ranks are preferentially placed on GPUs that are already relatively full but can still accommodate them; a request is admitted only when the entire stream fits and can meet its SLO when run alone, otherwise it is rejected; admitted requests are not migrated (Figure 11).
Figure 12 compares the maximum verified number of streams that can be served continuously by the two approaches with a fixed 24 GPUs at SLOs of 1000, 1200, and 1600 ms. Equal partitioning gives each stream an exclusive GPU group, with stages evenly partitioned and no batching; joint scheduling is the approach described above. After a stream begins outputting, it is counted only when every subsequent output latency remains within the SLO and the complete repeated test passes.
At an SLO of 1600 ms, the verified service-stream counts for equal partitioning and joint scheduling are, respectively: Wan2.1 1.3B, 12→39; Wan2.2 TI2V 5B, 24→46; Wan2.1 14B, 24→18; LTX-2.5 22B, 8→18; and MiniMax H3 33B, 6→10. Joint scheduling increases the number of concurrent streams for four of the models, by factors of 1.7–3.3; Wan2.1 14B decreases because its resource headroom is limited.
The benefit of joint scheduling mainly comes from reclaiming memory and compute time that cannot be shared across requests under equal partitioning. Models with lower single-stream resource consumption leave more idle capacity under equal partitioning and therefore benefit more; when the remaining resource headroom is small and a model is already close to the per-GPU capacity limit, the shared space is limited and the benefit correspondingly decreases. Specifically, when memory headroom is large, the scheduler can place more virtual ranks on the same GPU; when compute load is low, cross-stream batching and time-sharing can also fill previously idle time. For models with larger parameter counts, joint scheduling can additionally share block weights that are already resident and reduce weight replicas across requests; when both memory and compute are close to saturation, admitting another request may instead cause output latency to exceed the SLO.
Joint scheduling extends resource reuse from within one stream to across multiple requests: virtual ranks provide fine-grained task placement, cross-stream batching and time-sharing recover fragmented memory and compute capacity, and admission constraints are intended to protect the SLOs of running requests. It does not change the minimum single-stream output latency; instead, under a given GPU budget and output-latency requirement, it increases the number of requests the system can serve simultaneously. The actual benefit depends on the memory and compute headroom that different models can share.
Discussion and future work
To the best of our knowledge, PhysisServe is the first inference system that enables a 30B-scale autoregressive diffusion model to output continuously at a chunk interval below 10 ms while allowing multiple high-frequency tasks to share the same GPU pool. The pipeline can support more blocks and denoising steps by adding GPUs rather than directly extending the steady-state output interval. MiniMax H3 (50 blocks and 4 denoising steps) outputs one chunk every 4.1 ms on 200 GPUs, equivalent to approximately 1000 frames per second, excluding VAE decoding. Existing large real-time world models report tens of frames per second, although hardware and timing scopes differ. The multi-stream results use a separate 24-GPU configuration. Decoupling output latency from model scale creates room to reconcile the trade-off in which intelligence calls for a larger model while real-time operation calls for a smaller one—PhysisServe provides a feasible approach to this trade-off.
Decoupling output latency from model scale does not mean that inference for real-time world models has been fully solved. Our inference scheme still has limitations at both the algorithmic and system levels, which raises open questions for the community.
- Noisy KV cache management. PhysisServe’s pipelined denoising requires inference over noisy KV caches. Post-training algorithms for this paradigm are not yet mature, and the difference in generation quality and performance boundaries between noisy-KV-cache inference and clean-KV-cache inference remain unclear. We believe noisy-KV-cache inference can allow a model to combine a large parameter count with a high output frequency compared with clean-KV-cache inference, and can attract community attention and investment in developing noisy-KV-cache post-training algorithms.
- Response latency. This work attempts to solve output latency, not action response latency—the new action or observation still needs to pass through all stages in sequence before it can first appear in the output, so high-frequency closed-loop control remains subject to substantial limitations. We believe this limitation can be gradually mitigated through infrastructure-level techniques such as quantization, pruning, and model-architecture innovation, together with continued growth in chip compute.
Conclusion
The larger a world model is, the more accurate its understanding and prediction of the physical world may be; but the physical world places stringent demands on real-time operation, and once a model’s output frequency falls behind environmental changes, even an accurate prediction loses its value. For world models based on autoregressive denoising, model scale and output frequency have long been difficult to achieve simultaneously. An important reason is that existing inference schemes use a pipeline partitioning granularity that is too coarse, limiting parallelism and thereby constraining output frequency. This makes it difficult for world-model real-time performance to continue improving as compute increases, becoming a major bottleneck to deploying world models in real-world settings.
To overcome this bottleneck, PhysisServe refines the granularity of pipeline orchestration to the stage level (that is, one block at one denoising step) and makes three design choices:
- Stage-wise pipeline. Fine-grained pipeline orchestration makes single-stream output latency depend mainly on the computation and transfer of one stage.
- Time-sharing. Alternating execution and batching reduce the compute required by a single stream.
- Virtual rank. Multiple streams share one GPU pool, turning the idle capacity caused by rounding when GPUs are partitioned by stream into additional serviceable streams.
Experimental results show that PhysisServe can keep the single-stream output latency of models at different scales in the millisecond range: the minimum output latency across the five models is 1.895–7.547 ms, and the 33B MiniMax H3 reaches 4.083 ms, corresponding to approximately 1000 frames per second when each chunk decodes to 4 frames. The current results do not show a monotonic increase in minimum output latency with model parameter scale.
For multi-stream serving, joint scheduling can substantially improve concurrent service capacity for most models, but the benefit depends on model structure and resource headroom and is not uniform across all models. Overall, the experiments show that, through appropriate scheduling and allocation of GPU compute and memory, the trade-off between model scale and output frequency can still be mitigated when compute resources are limited.
Enabling sufficiently large models to keep their intelligence in step with the physical world, PhysisServe takes an exploratory step toward this goal.
Citation
Please cite this work as:
Physis AI Infra Team, "PhysisServe: Efficient Autoregressive Denoising Serving with Fine-Grained Stage-wise Pipelining," Physis AI, Oct. 12, 2026.
Or use the BibTeX citation:
@article{physisserve2026,
author = {Physis AI Infra Team},
title = {PhysisServe: Efficient Autoregressive Denoising Serving with Fine-Grained Stage-wise Pipelining},
journal = {Physis AI},
year = {2026},
month = {oct},
url = {https://physis-ai.com/blog/physisserve}
}
References
- , “A Brain Too Big to Carry — On-Device vs Datacenter Inference,” SemiAnalysis, Sep. 14, 2026. [Online]. Available: https://newsletter.semianalysis.com/p/a-brain-too-big-to-carry-on-device. Accessed: Oct. 13, 2026.
- , “Training Agents Inside of Scalable World Models,” arXiv preprint, 2025. arXiv:2509.24527.
- , “Genie 3: A New Frontier for World Models,” Google DeepMind blog, Aug. 5, 2025. [Online]. Available: https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/. Accessed: Oct. 13, 2026.
- , “World Action Models are Zero-shot Policies,” arXiv preprint, 2026. arXiv:2602.15922.
- , “Native Video-Action Pretraining for Generalizable Robot Control,” arXiv preprint, 2026. arXiv:2607.08639.
- , “StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation,” in Proceedings of Machine Learning and Systems, vol. 8, pp. 1656–1669, MLSys, 2026. [Online]. Available: https://proceedings.mlsys.org/paper_files/paper/2026/file/698cfaf72a208aef2e78bcac55b74328-Paper-Conference.pdf.
- , “Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data,” arXiv preprint, 2026. arXiv:2603.12686.
- , “Advancing Open-source World Models,” arXiv preprint, 2026. arXiv:2601.20540.
- , “MAGI-1: Autoregressive Video Generation at Scale,” arXiv preprint, 2025. arXiv:2505.13211.
- , “Introducing Helix 02: Full-Body Autonomy,” Company technical article, Jan. 27, 2026. [Online]. Available: https://www.figure.ai/news/helix-02. Accessed: Oct. 13, 2026.
- , “Wave Forcing: Towards Speed-of-Light Streaming Video Generation,” Project blog, Shanghai Jiao Tong University, 2026. [Online]. Available: https://sjtu-deng-lab.github.io/WaveForcing/. Accessed: Oct. 13, 2026.
- , “MiniMax H3,” Hugging Face model card, 2026. [Online]. Available: https://huggingface.co/MiniMaxAI/MiniMax-H3. Accessed: Oct. 13, 2026.
- , “Rolling Forcing: Autoregressive Long Video Diffusion in Real Time,” in International Conference on Learning Representations, vol. 2026, pp. 91177–91196, 2026. [Online]. Available: https://proceedings.iclr.cc/paper_files/paper/2026/file/935151cc6cb5d8b6816133b75233775a-Paper-Conference.pdf.
- , “Heuristics for Vector Bin Packing,” Unpublished manuscript, Jan. 2011. [Online]. Available: https://www.microsoft.com/en-us/research/publication/heuristics-for-vector-bin-packing/.
- , “Limited Discrepancy Search,” in Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, vol. 1, pp. 607–613, Morgan Kaufmann, 1995. [Online]. Available: https://www.ijcai.org/Proceedings/95-1/Papers/080.pdf.
