PyTorch Day China 2025

Hugging Face Hub is a premier platform for hosting and sharing machine learning models. It offers a comprehensive suite of features—from model discovery and hosting to deployment and collaboration—that enhance the accessibility and impact of your work. Join this session to explore how these capabilities can streamline your workflow and amplify the reach of your research.

Recent advances in reinforcement learning significantly boosts the reasoning capabilities of LLMs. Models such as OpenAI o3, DeepSeek r1, etc,. demonstrates magnificent performance in STEM and coding tasks. Yet, training such models requires complex infrastructures.
In this talk, we present verl (https://github.com/volcengine/verl), a comprehensive framework that utilizes HybridFlow programming abstraction to achieve both flexibility to implement various algorithms and high performance. verl has been adopted by various universities and companies for RL training, and is contributed by 100+ contributors from the community.
Through this talk, audiences will gain i) a basic understanding of various RL algorithms including GRPO; ii) best practices to implement tool calling and multi-turn rollout for agentic tasks, as well vision language model reasoning; iii) latest large scale performance optimization techniques for RL with MOE models such as DeepSeek v3.

Chinese PyTorch Community Overview & Resources
Provide a comprehensive introduction to the Chinese PyTorch community and its resources.

Community Events & Future Vision
Discuss plans to scale the community, drive technical innovation, and position PyTorch as the go-to framework for AI development in China.

Localized Tutorials & Documentation
Address the challenge of limited access to translated materials for PyTorch 2.x. Present ongoing efforts to translate official documentation and tutorials to empower Chinese learners.

Motivation
PyTorch supports a wide range of acceleration hardware beyond CPUs, including CUDA, XPU, MPS, NPU, HPU, and more. Its architecture allows new backend integration through two key components: ATen operators and device runtime.
While ATen operators are device-agnostic, the runtime remains device-specific, relying on APIs like torch.cuda and torch.xpu. This fragmentation complicates writing portable, hardware-agnostic code across PyTorch and its ecosystem.
To address this challenge, we propose torch.accelerator: a unified, device-agnostic runtime API for stream-based accelerators.
Design
An Accelerator refers to a device that collaborates with the CPU to accelerate computation, typically via asynchronous execution using Stream and Event for synchronization. Our design assumes a single active accelerator per host.
The torch.accelerator API provides a consistent interface for device and stream management, with backend support integrated via the existing DeviceGuardImplInterface registration mechanism.
Further Work
We are actively working on a unified device memory API. These will streamline the library, model, and UTs.
Reference
https://github.com/pytorch/pytorch/pull/132204

vLLM is a fast and easy-to-use library for LLM inference and serving. In this talk, I will briefly introduce the evolution of the vLLM project, the open-source community behind it, and highlight some features that are interesting to many users.

An introduction of torch_musa, which is built by Moore Threads to provide PyTorch backend support for MUSA architecture, and with its customized support for quantization, we built a light-weight model compression toolkit named NeuroTrim, which can quantize a variety of deep learning models and run it on native PyTorch.
We expanded the graph tracing capability of PyTorch by integrating fx graph, torchInductor, and ExportedIR, and added many lowered operators in torch_musa, which can be easily used for debugging, and then export to ONNX or other formats for custom accelerator hardwares.

Video generation models have emerged as powerful tools, enabling compelling multimedia content creation across various applications. At ByteDance, we tackled the substantial challenge of efficiently scaling large-scale video generation model training to thousands of GPUs using PyTorch's robust software ecosystem.

In this talk, we will present several key innovations enabled by PyTorch to significantly enhance training efficiency:
1. Efficient Kernel Fusion with PyTorch Compile for optimized GPU computations.
2. Large-Model Training enabled by PyTorch's Checkpointing and Offloading techniques to address memory constraints.
3. Scalable Distributed Training combining Sequence Parallelism (SP) and Hybrid Sharding Data Parallelism (HSDP) for near-linear GPU scalability.
4. Load-Balancing Strategy dynamically distributing workloads for variable-length videos to eliminate performance bottlenecks.

Our session will delve into the practical methodologies, share crucial insights gained, and provide actionable best practices, empowering attendees to optimize their own large-scale model training workflows using PyTorch.

torch dynamo is the core compilation optimization component in pytorch 2.0. torch dynamo can optimize dynamic graphs with minor modifications, and fall back to the default implementation when it is not supported, which greatly reduces the difficulty of enabling compilation optimization.

However, in practice, due to the diversity of AI workloads, there is still a certain threshold for enabling torch dynamo. To address these issues, we have optimized torch.compile to make it easier for users of different models (LLM/others) and loads (inference/training) to benefit from compilation optimization.

Deploying PyTorch in large-scale heterogeneous computing have huge challenges in stability, usability, performance, and quality.

This session will showcase best practices using real-world Huawei Ascend application scenarios and community contributions, demonstrating PyTorch and ecosystem components in large-scale heterogeneous computing.

Through Ascend-optimized technologies and architecture improvements in PyTorch, we have greatly enhanced computing performance in traning and inference, meeting production standards.

We’ve proposed and implemented features for heterogeneous compute and accelerator quality standards in PyTorch. Driving PyTorch’s application in heterogeneous compute, like PyTorch ecosystem projects: DeepSpeed, vLLM.

Finally, we will share real user stories from large-scale training and distributed inference, presenting best practices on the Ascend platform and helping the community address challenges in diverse compute scenarios.

Galvatron is a PyTorch-native, open-source framework for the efficient distributed training of large-scale Transformer models, with specialized optimizations for automatic hybrid parallelism strategies. Given a Transformer model, Galvatron first employs PyTorch profiler to analyze the model execution workload characteristics, creating a precise cost model. Then, Galvatron uses decision trees and dynamic programming to automatically deduce the best combination of parallelism dimensions for each model layer, covering data, tensor, pipeline, sharded data, sequence parallelism, and recomputation. Finally, Galvatron leverages PyTorch features like FSDP and checkpointing—enjoying its seamless integration with various accelerators like NVIDIA GPUs and Ascend NPUs—to deploy and train the model. As an open-source project with comprehensive documentation, Galvatron is designed to be user-friendly, enabling easy integration with minimal code changes. Collaborations from both academia and industry, such as BAAI, Huawei, and ByteDance, highlight its practical applications and superior efficiency compared to existing frameworks. Discover more at https://github.com/PKU-DAIR/Hetu-Galvatron.

PyTorch is one of the most popular frameworks for deep learning and machine learning, and Intel has been a long-term contributor and advocate for the PyTorch community. In this talk, we will share our experience in contributing to PyTorch in the core framework and its ecosystem. We will show how to make AI applications more popular through the improvement of hardware computing power and the optimization of open source software. We will introduce the latest relevant progress on Intel GPUs on PyTorch, and the low-cost deployment solution of DeepSeek R1 671B full-blooded version on Intel Xeon CPU. We will also introduce some PyTorch ecosystem projects that we have participated in in the past, such as HuggingFace, vLLM, SGLang, etc. Finally, we will discuss future plans and visions and continue to work with the PyTorch Foundation to drive deep learning and machine learning in a better direction.

In the past two years, the Triton language has become the second most popular AI operator development language in the industry after CUDA C. Although there is still a gap in popularity compared to CUDA C, due to its open-source compiler ecosystem.
However, the current compilers supporting the Triton language still fail to support multiple hardware architectures in the main community. As a result, each hardware manufacturer has to maintain its own version of the Triton compiler separately, which brings new difficulties to upper - layer users, such as inconsistent compiler versions and functional differences.
To address this issue, we have created the FlagTree open - source project and ecological community that support the Triton language. We are committed to building an open - source and unified AI compiler for a variety of AI chips. By unifying the multi - backend compiler, we aim to build an ecosystem that can support other chip hardware architectures (such as GPGPU, DSA, and RISC - V AI), thus enabling cross - platform operation.

SGLang is an open-source Large Language Model (LLM) inference system that is highly efficient and widely adopted by many companies like xAI, Nvidia and AMD. In this session, I will introduce some key features of SGLang, including the design and implementation of PD disaggregation, large-scale expert parallelism and data parallelism for DeepSeek models, hierarchical KV cache offloading, and highly efficient speculative decoding. I will also share some insights into the future development of the SGLang community.