History

zhangyue 985d0dee5f Kunlun dist op (#225 ) * kunlun dist inference fix * kunlun distributed * 添加昆仑芯分布式脚本以及解决运行llama遇到的问题 * set -j8 * format * move run_pytorch.py int o cuda/ * update notes --------- Co-authored-by: weijie01 <weijie01@baidu.com> Co-authored-by: wanghailu <wanghailu0717@163.com> Co-authored-by: Haojie Wang <haojie0429@gmail.com>		2024-04-23 15:46:25 +08:00
..
bang	Kunlun dist op (#225 )	2024-04-23 15:46:25 +08:00
cuda	Kunlun dist op (#225 )	2024-04-23 15:46:25 +08:00
kunlun	Kunlun dist op (#225 )	2024-04-23 15:46:25 +08:00
README.md	针对bert和gpt2模型分布式推理的优化 (#221 )	2024-04-01 14:04:28 +08:00
__init__.py	Kunlun dist op (#225 )	2024-04-23 15:46:25 +08:00
parallel.py	impl distributed launch with NCCL (#106 )	2023-09-05 09:47:35 +08:00
parallel_opt.py	Kunlun dist op (#225 )	2024-04-23 15:46:25 +08:00
placement.py	tensor parallel for transformer (#125 )	2023-09-14 14:19:45 +08:00

分布式脚本

使用 --export_onnx 设置导出onnx的目录，默认为当前路径 ./，不使用这个flag则只进行计算和生成输入输出。

python run_pytorch.py --model gpt2  --batch_size 1  --length 1 --export_onnx ./

会在当前目录下生成输入输出文件test_inputs.npy 和 test_results.npy，目前只支持单一输入输出。

python cuda_launch.py --model "/XXX/XXX.onnx" --nproc_per_node 4