AI Infra Intern, Parallel Computing & Inference
Develop and maintain the vLLM_MLU and SGLang_MLU inference engines for Cambricon MLUs, spanning model enablement, scheduling and cache systems, kernel optimization, accuracy testing, and debugging.
Work and outcomes
- Enabled and optimized Qwen3.5-and-later and KimiK models, including Prefix Caching, Chunked Prefill, and All-to-All; improved scheduling, refactored cache management, and supported migration from PyTorch to the native MLU backend.
- Used Torch Profiler to inspect scheduling paths and performance traces, and GenCase to capture kernel shapes; fixed regressions, repeated output, and accuracy issues across long-context, multimodal, and agent workloads.
- Enabled Qwen3.5 on SGLang_MLU through GDN linear-attention routing, scheduling, and Triton kernels, then integrated and validated MTP speculative decoding to reduce end-to-end generation latency.
- Evaluated Beam Search for multi-candidate generation and implemented a baseline; developed BangC kernels and fixed issues in TransformerEngine training infrastructure.