应用介绍

*Pangu72B支撑RL练习

支撑模型列表

模型 硬件 精度类型 安排形态 openPangu-Ultra-MoE-718B A3 INT8 PD分别 openPangu-Ultra-MoE-718B A2 INT8 PD分别 openPangu-72B A3 INT8 PD分别 openPangu-38B A3 INT8 混布 openPangu-38B A2 INT8 混布 openPangu-7B A3 BF16 混布 openPangu-7B A2 BF16 混布 openPangu-7BVL A3 BF16 混布 DeepSeek-R1 A3 INT8 PD分别 DeepSeek-R1 A3 W4A8C16 PD分别 DeepSeek-R1 A3 BF16 PD分别 DeepSeek-R1 A2 INT8 PD分别 DeepSeek-V3.1 A3 INT8 PD分别 DeepSeek-V3.2 A3 INT8 PD分别 DeepSeek-OCR A2 BF16 混布 Qwen2.5-7B A3 INT8 混布（TP>=1 DP=1） Qwen2.5-7B A2 INT8 混布（TP>=1 DP=1） QwQ A3 BF16 PD分别 QwQ A2 BF16 PD分别 Qwen3-235B A3 INT8 PD分别 Qwen3-235B A2 BF16 PD分别 Qwen3-32B A3 BF16 PD分别 Qwen3-32B A3 INT8 PD分别 Qwen3-30B A3 BF16 PD分别 Kimi-K2 A3 W4A8C16 PD分别 Kimi-K2 Thinking A3 W4A8C16 PD分别 Longcat-flash A3 BF16 PD分别 Ling-1T A3 BF16 PD分别 GPT-OSS120B A3 INT8 PD分别 GPT-OSS120B A2 INT8 PD分别 GPT-OSS20B A3 INT8 PD分别 GPT-OSS20B A2 INT8 PD分别

安装包

硬件 架构 镜像文件 Tar包 A3 arm docker pull swr.cn-east-4.myhuaweicloud.com/omni-ci/omniinfer-a3-arm:release_v0.8.0-vllm omni_infer-a3-arm:v0.8.0_vllm A3 x86 docker pull swr.cn-east-4.myhuaweicloud.com/omni-ci/omniinfer-a3-x86:release_v0.8.0-vllm omni_infer-a3-x86:v0.8.0_vllm A2 arm docker pull swr.cn-east-4.myhuaweicloud.com/omni-ci/omniinfer-a2-arm:release_v0.8.0-vllm omni_infer-a2-arm:v0.8.0_vllm A2 x86 docker pull swr.cn-east-4.myhuaweicloud.com/omni-ci/omniinfer-a2-x86:release_v0.8.0-vllm omni_infer-a2-x86:v0.8.0_vllm

Omni-Infer v0.8.0 已经宣布，超大年夜范围 MoE 模型推理加快技巧。

v0.8.0

核心特点

详情查看：https://gitee.com/omniai/omniinfer/releases/v0.8.0