add NPU docker images

This commit is contained in:
hiyouga 2024-05-15 19:20:11 +08:00
parent 75f405ec30
commit e1f4e53915
2 changed files with 11 additions and 1 deletions

View File

@ -342,7 +342,7 @@ To enable FlashAttention-2 on the Windows platform, you need to install the prec
<details><summary>For Ascend NPU users</summary> <details><summary>For Ascend NPU users</summary>
To utilize Ascend NPU devices for (distributed) training and inference, you need to install the **[torch-npu](https://gitee.com/ascend/pytorch)** package and the **[Ascend CANN Kernels](https://www.hiascend.com/developer/download/community/result?module=cann)**. To utilize Ascend NPU devices for (distributed) training and inference, you need to install the **[torch-npu](https://gitee.com/ascend/pytorch)** library and the **[Ascend CANN Kernels](https://www.hiascend.com/developer/download/community/result?module=cann)**.
| Requirement | Minimum | Recommend | | Requirement | Minimum | Recommend |
| ------------ | ------- | --------- | | ------------ | ------- | --------- |
@ -351,6 +351,11 @@ To utilize Ascend NPU devices for (distributed) training and inference, you need
| torch-npu | 2.2.0 | 2.2.0 | | torch-npu | 2.2.0 | 2.2.0 |
| deepspeed | 0.13.2 | 0.13.2 | | deepspeed | 0.13.2 | 0.13.2 |
Docker image:
- 32GB: [Download page](http://mirrors.cn-central-221.ovaijisuan.com/detail/130.html)
- 64GB: Coming soon
Remember to use `ASCEND_RT_VISIBLE_DEVICES` instead of `CUDA_VISIBLE_DEVICES` to specify the device to use. Remember to use `ASCEND_RT_VISIBLE_DEVICES` instead of `CUDA_VISIBLE_DEVICES` to specify the device to use.
If you cannot infer model on NPU devices, try setting `do_sample: false` in the configurations. If you cannot infer model on NPU devices, try setting `do_sample: false` in the configurations.

View File

@ -351,6 +351,11 @@ pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/downl
| torch-npu | 2.2.0 | 2.2.0 | | torch-npu | 2.2.0 | 2.2.0 |
| deepspeed | 0.13.2 | 0.13.2 | | deepspeed | 0.13.2 | 0.13.2 |
Docker 镜像:
- 32GB[下载地址](http://mirrors.cn-central-221.ovaijisuan.com/detail/130.html)
- 64GB敬请期待
请记得使用 `ASCEND_RT_VISIBLE_DEVICES` 而非 `CUDA_VISIBLE_DEVICES` 来指定您使用的设备。 请记得使用 `ASCEND_RT_VISIBLE_DEVICES` 而非 `CUDA_VISIBLE_DEVICES` 来指定您使用的设备。
如果遇到无法正常推理的情况,请尝试设置 `do_sample: false` 如果遇到无法正常推理的情况,请尝试设置 `do_sample: false`