p04798526
/
LLaMA-Factory-Mirror
1
0
Fork 1
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update docs

This commit is contained in:
hiyouga 2024-05-06 21:47:00 +08:00
parent a34f526f10
commit 34d33e2257
44 changed files with 487 additions and 516 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

69
README.md
View File

@ -276,18 +276,19 @@ huggingface-cli login
| ------------ | ------- | --------- |
| python | 3.8 | 3.10 |
| torch | 1.13.1 | 2.2.0 |
| transformers | 4.37.2 | 4.39.3 |
| datasets | 2.14.3 | 2.18.0 |
| accelerate | 0.27.2 | 0.28.0 |
| transformers | 4.37.2 | 4.40.1 |
| datasets | 2.14.3 | 2.19.1 |
| accelerate | 0.27.2 | 0.30.0 |
| peft | 0.9.0 | 0.10.0 |
| trl | 0.8.1 | 0.8.1 |
| trl | 0.8.1 | 0.8.6 |
| Optional | Minimum | Recommend |
| ------------ | ------- | --------- |
| CUDA | 11.6 | 12.2 |
| deepspeed | 0.10.0 | 0.14.0 |
| bitsandbytes | 0.39.0 | 0.43.0 |
| flash-attn | 2.3.0 | 2.5.6 |
| bitsandbytes | 0.39.0 | 0.43.1 |
| vllm | 0.4.0 | 0.4.2 |
| flash-attn | 2.3.0 | 2.5.8 |
### Hardware Requirement
@ -305,24 +306,15 @@ huggingface-cli login
## Getting Started
### Data Preparation
Please refer to [data/README.md](data/README.md) for checking the details about the format of dataset files. You can either use datasets on HuggingFace / ModelScope hub or load the dataset in local disk.
> [!NOTE]
> Please update `data/dataset_info.json` to use your custom dataset.
### Dependence Installation
### Installation
```bash
git clone https://github.com/hiyouga/LLaMA-Factory.git
conda create -n llama_factory python=3.10
conda activate llama_factory
cd LLaMA-Factory
pip install -e .[metrics]
```
Extra dependencies available: deepspeed, metrics, galore, badam, vllm, bitsandbytes, gptq, awq, aqlm, qwen, modelscope, quality
Extra dependencies available: metrics, deepspeed, bitsandbytes, vllm, galore, badam, gptq, awq, aqlm, qwen, modelscope, quality
<details><summary>For Windows users</summary>
@ -336,19 +328,41 @@ To enable FlashAttention-2 on the Windows platform, you need to install the prec
</details>
### Train with LLaMA Board GUI (powered by [Gradio](https://github.com/gradio-app/gradio))
### Data Preparation
Please refer to [data/README.md](data/README.md) for checking the details about the format of dataset files. You can either use datasets on HuggingFace / ModelScope hub or load the dataset in local disk.
> [!NOTE]
> Please update `data/dataset_info.json` to use your custom dataset.
### Quickstart
The following 3 commands conduct LoRA fine-tuning, inference and merging for Llama3-8B-Instruct model, respectively.
```bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_sft.yaml
CUDA_VISIBLE_DEVICES=0 llamafactory-cli chat examples/inference/llama3_lora_sft.yaml
CUDA_VISIBLE_DEVICES=0 llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml
```
See [examples/README.md](examples/README.md) for advanced usage.
> [!TIP]
> Use `llamafactory-cli help` to show help information.
### Use LLaMA Board GUI (powered by [Gradio](https://github.com/gradio-app/gradio))
> [!IMPORTANT]
> LLaMA Board GUI only supports training on a single GPU, please use [CLI](#train-with-command-line-interface) for distributed training.
> LLaMA Board GUI only supports training on a single GPU.
#### Use local environment
```bash
llamafactory-cli webui
CUDA_VISIBLE_DEVICES=0 llamafactory-cli webui
```
> [!TIP]
> To modify the default setting in the LLaMA Board GUI, you can use environment variables, e.g., `export CUDA_VISIBLE_DEVICES=0 GRADIO_SERVER_NAME=0.0.0.0 GRADIO_SERVER_PORT=7860 GRADIO_SHARE=False` (use `set` command on Windows OS).
> To modify the default setting in the LLaMA Board GUI, you can use environment variables, e.g., `export GRADIO_SERVER_NAME=0.0.0.0 GRADIO_SERVER_PORT=7860 GRADIO_SHARE=False` (use `set` command on Windows OS).
<details><summary>For Alibaba Cloud users</summary>
@ -389,21 +403,10 @@ docker compose -f ./docker-compose.yml up -d
</details>
### Train with Command Line Interface
See [examples/README.md](examples/README.md) for usage.
> [!TIP]
> Use `llamafactory-cli train -h` to display arguments description.
### Deploy with OpenAI-style API and vLLM
```bash
CUDA_VISIBLE_DEVICES=0,1 API_PORT=8000 llamafactory-cli api \
--model_name_or_path meta-llama/Meta-Llama-3-8B-Instruct \
--template llama3 \
--infer_backend vllm \
--vllm_enforce_eager
CUDA_VISIBLE_DEVICES=0,1 API_PORT=8000 llamafactory-cli api examples/inference/llama3_vllm.yaml
```
### Download from ModelScope Hub

71
README_zh.md
View File

@ -163,7 +163,7 @@ https://github.com/hiyouga/LLaMA-Factory/assets/16256802/ec36a9dd-37f4-4f72-81bd
| [Yuan](https://huggingface.co/IEITYuan) | 2B/51B/102B | q_proj,v_proj | yuan |
> [!NOTE]
> **默认模块**应作为 `--lora_target` 参数的默认值，可使用 `--lora_target all` 参数指定全部模块以得到更好的效果。
> **默认模块**应作为 `--lora_target` 参数的默认值，可使用 `--lora_target all` 参数指定全部模块以取得更好的效果。
>
> 对于所有“基座”（Base）模型，`--template` 参数可以是 `default`, `alpaca`, `vicuna` 等任意值。但“对话”（Instruct/Chat）模型请务必使用**对应的模板**。
>
@ -276,18 +276,19 @@ huggingface-cli login
| ------------ | ------- | --------- |
| python | 3.8 | 3.10 |
| torch | 1.13.1 | 2.2.0 |
| transformers | 4.37.2 | 4.39.3 |
| datasets | 2.14.3 | 2.18.0 |
| accelerate | 0.27.2 | 0.28.0 |
| transformers | 4.37.2 | 4.40.1 |
| datasets | 2.14.3 | 2.19.1 |
| accelerate | 0.27.2 | 0.30.0 |
| peft | 0.9.0 | 0.10.0 |
| trl | 0.8.1 | 0.8.1 |
| trl | 0.8.1 | 0.8.6 |
| 可选项 | 至少 | 推荐 |
| ------------ | ------- | --------- |
| CUDA | 11.6 | 12.2 |
| deepspeed | 0.10.0 | 0.14.0 |
| bitsandbytes | 0.39.0 | 0.43.0 |
| flash-attn | 2.3.0 | 2.5.6 |
| bitsandbytes | 0.39.0 | 0.43.1 |
| vllm | 0.4.0 | 0.4.2 |
| flash-attn | 2.3.0 | 2.5.8 |
### 硬件依赖
@ -305,24 +306,15 @@ huggingface-cli login
## 如何使用
### 数据准备
关于数据集文件的格式，请参考 [data/README_zh.md](data/README_zh.md) 的内容。你可以使用 HuggingFace / ModelScope 上的数据集或加载本地数据集。
> [!NOTE]
> 使用自定义数据集时，请更新 `data/dataset_info.json` 文件。
### 安装依赖
### 安装 LLaMA Factory
```bash
git clone https://github.com/hiyouga/LLaMA-Factory.git
conda create -n llama_factory python=3.10
conda activate llama_factory
cd LLaMA-Factory
pip install -e .[metrics]
```
可选的额外依赖项：deepspeed、metrics、galore、badam、vllm、bitsandbytes、gptq、awq、aqlm、qwen、modelscope、quality
可选的额外依赖项：metrics、deepspeed、bitsandbytes、vllm、galore、badam、gptq、awq、aqlm、qwen、modelscope、quality
<details><summary>Windows 用户指南</summary>
@ -336,19 +328,41 @@ pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/downl
</details>
### 利用 LLaMA Board 可视化界面训练（由 [Gradio](https://github.com/gradio-app/gradio) 驱动）
### 数据准备
关于数据集文件的格式，请参考 [data/README_zh.md](data/README_zh.md) 的内容。你可以使用 HuggingFace / ModelScope 上的数据集或加载本地数据集。
> [!NOTE]
> 使用自定义数据集时，请更新 `data/dataset_info.json` 文件。
### 快速开始
下面三行命令分别对 Llama3-8B-Instruct 模型进行 LoRA 微调、推理和合并。
```bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_sft.yaml
CUDA_VISIBLE_DEVICES=0 llamafactory-cli chat examples/inference/llama3_lora_sft.yaml
CUDA_VISIBLE_DEVICES=0 llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml
```
高级用法请参考 [examples/README_zh.md](examples/README_zh.md)。
> [!TIP]
> 使用 `llamafactory-cli help` 显示使用帮助。
### 使用 LLaMA Board 可视化界面（由 [Gradio](https://github.com/gradio-app/gradio) 驱动）
> [!IMPORTANT]
> LLaMA Board 可视化界面目前仅支持单 GPU 训练，请使用[命令行接口](#利用命令行接口训练)来进行多 GPU 分布式训练。
> LLaMA Board 可视化界面目前仅支持单 GPU 训练。
#### 使用本地环境
```bash
llamafactory-cli webui
CUDA_VISIBLE_DEVICES=0 llamafactory-cli webui
```
> [!TIP]
> 您可以使用环境变量来修改 LLaMA Board 可视化界面的默认设置，例如 `export CUDA_VISIBLE_DEVICES=0 GRADIO_SERVER_NAME=0.0.0.0 GRADIO_SERVER_PORT=7860 GRADIO_SHARE=False`（Windows 系统可使用 `set` 指令）。
> 您可以使用环境变量来修改 LLaMA Board 可视化界面的默认设置，例如 `export GRADIO_SERVER_NAME=0.0.0.0 GRADIO_SERVER_PORT=7860 GRADIO_SHARE=False`（Windows 系统可使用 `set` 指令）。
<details><summary>阿里云用户指南</summary>
@ -389,21 +403,10 @@ docker compose -f ./docker-compose.yml up -d
</details>
### 利用命令行接口训练
使用方法请参考 [examples/README_zh.md](examples/README_zh.md)。
> [!TIP]
> 您可以执行 `llamafactory-cli train -h` 来查看参数文档。
### 利用 vLLM 部署 OpenAI API
```bash
CUDA_VISIBLE_DEVICES=0,1 API_PORT=8000 llamafactory-cli api \
--model_name_or_path meta-llama/Meta-Llama-3-8B-Instruct \
--template llama3 \
--infer_backend vllm \
--vllm_enforce_eager
CUDA_VISIBLE_DEVICES=0,1 API_PORT=8000 llamafactory-cli api examples/inference/llama3_vllm.yaml
```
### 从魔搭社区下载

9
examples/README.md
View File

@ -1,9 +1,16 @@
We provide diverse examples about fine-tuning LLMs.
```bash
export CUDA_VISIBLE_DEVICES=0
cd examples/lora_single_gpu
llamafactory-cli train llama3_lora_pretrain.yaml # Do continuous pre-training using LoRA
```
```
examples/
├── lora_single_gpu/
│ ├── pretrain.sh: Do continuous pre-training using LoRA
│ ├── `
│ ├── sft.sh: Do supervised fine-tuning using LoRA
│ ├── reward.sh: Do reward modeling using LoRA
│ ├── ppo.sh: Do PPO training using LoRA

2
examples/extras/badam/sft.sh
View File

@ -10,7 +10,7 @@ CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--finetuning_type full \
--use_badam \
--badam_switch_mode descending \
--badam_switch_interval 50 \
--badam_switch_block_every 50 \
--badam_verbose 2 \
--output_dir ../../../saves/LLaMA2-7B/badam/sft \
--overwrite_cache \

7
examples/inference/api_demo.sh
View File

@ -1,7 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 API_PORT=8000 llamafactory-cli api \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft \
--template default \
--finetuning_type lora

7
examples/inference/cli_demo.sh
View File

@ -1,7 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli chat \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft \
--template default \
--finetuning_type lora

12
examples/inference/evaluate.sh
View File

@ -1,12 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli eval \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft \
--template fewshot \
--finetuning_type lora \
--task mmlu \
--split test \
--lang en \
--n_shot 5 \
--batch_size 4

2
examples/inference/llama3.yaml Normal file
View File

@ -0,0 +1,2 @@
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
template: llama3

4
examples/inference/llama3_lora_sft.yaml Normal file
View File

@ -0,0 +1,4 @@
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
adapter_name_or_path: saves/llama3-8b/lora/sft
template: llama3
finetuning_type: lora

4
examples/inference/llama3_vllm.yaml Normal file
View File

@ -0,0 +1,4 @@
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
template: llama3
infer_backend: vllm
vllm_enforce_eager: true

8
examples/inference/web_demo.sh
View File

@ -1,8 +0,0 @@
#!/bin/bash
# add `--visual_inputs True` to load MLLM
CUDA_VISIBLE_DEVICES=0 llamafactory-cli webchat \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft \
--template default \
--finetuning_type lora

35
examples/lora_single_gpu/dpo.sh
View File

@ -1,35 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage dpo \
--do_train \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft \
--create_new_adapter \
--dataset orca_rlhf \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/dpo \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--warmup_steps 20 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 1e-5 \
--num_train_epochs 1.0 \
--max_samples 1000 \
--val_size 0.1 \
--dpo_ftx 1.0 \
--plot_loss \
--fp16

39
examples/lora_single_gpu/llama3_lora_dpo.yaml Normal file
View File

@ -0,0 +1,39 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
# method
stage: dpo
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
dpo_ftx: 1.0
# dataset
dataset: orca_rlhf
template: llama3
cutoff_len: 1024
max_samples: 1000
val_size: 0.1
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/llama3-8b/lora/dpo
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
# train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 0.00001
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_steps: 0.1
fp16: true
# eval
per_device_eval_batch_size: 1
evaluation_strategy: steps
eval_steps: 500

19
examples/lora_single_gpu/llama3_lora_eval.yaml Normal file
View File

@ -0,0 +1,19 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
adapter_name_or_path: saves/llama3-8b/lora/sft
# method
finetuning_type: lora
# dataset
task: mmlu
split: test
template: fewshot
lang: en
n_shot: 5
# output
save_dir: saves/llama3-8b/lora/eval
# eval
batch_size: 4

38
examples/lora_single_gpu/llama3_lora_orpo.yaml Normal file
View File

@ -0,0 +1,38 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
# method
stage: orpo
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
# dataset
dataset: orca_rlhf
template: llama3
cutoff_len: 1024
max_samples: 1000
val_size: 0.1
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/llama3-8b/lora/orpo
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
# train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 0.00001
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_steps: 0.1
fp16: true
# eval
per_device_eval_batch_size: 1
evaluation_strategy: steps
eval_steps: 500

38
examples/lora_single_gpu/llama3_lora_ppo.yaml Normal file
View File

@ -0,0 +1,38 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
reward_model: saves/llama3-8b/lora/reward
# method
stage: ppo
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
# dataset
dataset: identity,alpaca_gpt4_en
template: llama3
cutoff_len: 1024
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/llama3-8b/lora/ppo
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
# train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 0.00001
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_steps: 0.1
fp16: true
# generate
max_new_tokens: 512
top_k: 0
top_p: 0.9

24
examples/lora_single_gpu/llama3_lora_predict.yaml Normal file
View File

@ -0,0 +1,24 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
adapter_name_or_path: saves/llama3-8b/lora/sft
# method
stage: sft
do_predict: true
finetuning_type: lora
# dataset
dataset: identity,alpaca_gpt4_en
template: llama3
cutoff_len: 1024
max_samples: 50
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/llama3-8b/lora/predict
overwrite_output_dir: true
# eval
per_device_eval_batch_size: 1
predict_with_generate: true

37
examples/lora_single_gpu/llama3_lora_pretrain.yaml Normal file
View File

@ -0,0 +1,37 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
# method
stage: pt
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
# dataset
dataset: c4_demo
cutoff_len: 1024
max_samples: 1000
val_size: 0.1
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/llama3-8b/lora/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
# train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 0.0001
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_steps: 0.1
fp16: true
# eval
per_device_eval_batch_size: 1
evaluation_strategy: steps
eval_steps: 500

38
examples/lora_single_gpu/llama3_lora_reward.yaml Normal file
View File

@ -0,0 +1,38 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
# method
stage: rm
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
# dataset
dataset: orca_rlhf
template: llama3
cutoff_len: 1024
max_samples: 1000
val_size: 0.1
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/llama3-8b/lora/reward
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
# train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 0.00001
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_steps: 0.1
fp16: true
# eval
per_device_eval_batch_size: 1
evaluation_strategy: steps
eval_steps: 500

38
examples/lora_single_gpu/llama3_lora_sft.yaml Normal file
View File

@ -0,0 +1,38 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
# method
stage: sft
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
# dataset
dataset: identity,alpaca_gpt4_en
template: llama3
cutoff_len: 1024
max_samples: 1000
val_size: 0.1
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/llama3-8b/lora/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
# train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 0.0001
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_steps: 0.1
fp16: true
# eval
per_device_eval_batch_size: 1
evaluation_strategy: steps
eval_steps: 500

22
examples/lora_single_gpu/llama3_preprocess.yaml Normal file
View File

@ -0,0 +1,22 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
# method
stage: sft
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
# dataset
dataset: identity,alpaca_gpt4_en
template: llama3
cutoff_len: 1024
max_samples: 1000
val_size: 0.1
overwrite_cache: true
preprocessing_num_workers: 16
tokenized_path: saves/llama3-8b/dataset/sft # use `tokenized_path` in config to load data
# output
output_dir: saves/llama3-8b/lora/sft
overwrite_output_dir: true

39
examples/lora_single_gpu/llava1_5_lora_sft.yaml Normal file
View File

@ -0,0 +1,39 @@
# model
model_name_or_path: llava-hf/llava-1.5-7b-hf
visual_inputs: true
# method
stage: sft
do_train: true
finetuning_type: lora
lora_target: q_proj,v_proj
# dataset
dataset: mllm_demo
template: vicuna
cutoff_len: 1024
max_samples: 1000
val_size: 0.1
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/llava1_5-7b/lora/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
# train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 0.0001
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_steps: 0.1
fp16: true
# eval
per_device_eval_batch_size: 1
evaluation_strategy: steps
eval_steps: 500

32
examples/lora_single_gpu/orpo.sh
View File

@ -1,32 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage orpo \
--do_train \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--dataset orca_rlhf \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/orpo \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--warmup_steps 20 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 1e-5 \
--num_train_epochs 1.0 \
--max_samples 1000 \
--val_size 0.1 \
--plot_loss \
--fp16

32
examples/lora_single_gpu/ppo.sh
View File

@ -1,32 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage ppo \
--do_train \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft \
--create_new_adapter \
--dataset alpaca_gpt4_en \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--reward_model ../../saves/LLaMA2-7B/lora/reward \
--output_dir ../../saves/LLaMA2-7B/lora/ppo \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 512 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--save_steps 100 \
--learning_rate 1e-5 \
--num_train_epochs 1.0 \
--max_samples 1000 \
--top_k 0 \
--top_p 0.9 \
--max_new_tokens 256 \
--plot_loss \
--fp16

19
examples/lora_single_gpu/predict.sh
View File

@ -1,19 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage sft \
--do_predict \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft,../../saves/LLaMA2-7B/lora/dpo \
--dataset alpaca_gpt4_en,glaive_toolcall \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--output_dir ../../saves/LLaMA2-7B/lora/predict \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--per_device_eval_batch_size 1 \
--max_samples 20 \
--predict_with_generate

19
examples/lora_single_gpu/prepare.sh
View File

@ -1,19 +0,0 @@
#!/bin/bash
# use `--tokenized_path` in training script to load data
CUDA_VISIBLE_DEVICES= llamafactory-cli train \
--stage sft \
--do_train \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--dataset alpaca_gpt4_en,glaive_toolcall \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/sft \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--max_samples 3000 \
--tokenized_path ../../saves/datasets/sft

31
examples/lora_single_gpu/pretrain.sh
View File

@ -1,31 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage pt \
--do_train \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--dataset c4_demo \
--dataset_dir ../../data \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/pretrain \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--warmup_steps 20 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 5e-5 \
--num_train_epochs 3.0 \
--max_samples 10000 \
--val_size 0.1 \
--plot_loss \
--fp16

33
examples/lora_single_gpu/reward.sh
View File

@ -1,33 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage rm \
--do_train \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft \
--create_new_adapter \
--dataset orca_rlhf \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/reward \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--warmup_steps 20 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--learning_rate 1e-5 \
--num_train_epochs 1.0 \
--max_samples 5000 \
--val_size 0.1 \
--plot_loss \
--fp16

32
examples/lora_single_gpu/sft.sh
View File

@ -1,32 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage sft \
--do_train \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--dataset alpaca_gpt4_en,glaive_toolcall \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/sft \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--warmup_steps 20 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 5e-5 \
--num_train_epochs 3.0 \
--max_samples 3000 \
--val_size 0.1 \
--plot_loss \
--fp16

33
examples/lora_single_gpu/sft_mllm.sh
View File

@ -1,33 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage sft \
--do_train \
--model_name_or_path llava-hf/llava-1.5-7b-hf \
--visual_inputs \
--dataset mllm_demo \
--dataset_dir ../../data \
--template vicuna \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/sft_mllm \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--warmup_steps 20 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 5e-5 \
--num_train_epochs 100.0 \
--max_samples 3000 \
--val_size 0.1 \
--plot_loss \
--fp16

11
examples/merge_lora/llama3_gptq.yaml Normal file
View File

@ -0,0 +1,11 @@
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
template: llama3
# export
export_dir: models/llama3_gptq
export_quantization_bit: 4
export_quantization_dataset: data/c4_demo.json
export_size: 2
export_device: cpu
export_legacy_format: false

13
examples/merge_lora/llama3_lora_sft.yaml Normal file
View File

@ -0,0 +1,13 @@
# Note: DO NOT use quantized model or quantization_bit when merging lora weights
# model
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
adapter_name_or_path: saves/llama3-8b/lora/sft
template: llama3
finetuning_type: lora
# export
export_dir: models/llama3_lora_sft
export_size: 2
export_device: cpu
export_legacy_format: false

12
examples/merge_lora/merge.sh
View File

@ -1,12 +0,0 @@
#!/bin/bash
# DO NOT use quantized model or quantization_bit when merging lora weights
CUDA_VISIBLE_DEVICES=0 llamafactory-cli export \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--adapter_name_or_path ../../saves/LLaMA2-7B/lora/sft \
--template default \
--finetuning_type lora \
--export_dir ../../models/llama2-7b-sft \
--export_size 2 \
--export_device cpu \
--export_legacy_format False

11
examples/merge_lora/quantize.sh
View File

@ -1,11 +0,0 @@
#!/bin/bash
# NEED TO run `merge.sh` before using this script
CUDA_VISIBLE_DEVICES=0 llamafactory-cli export \
--model_name_or_path ../../models/llama2-7b-sft \
--template default \
--export_dir ../../models/llama2-7b-sft-int4 \
--export_quantization_bit 4 \
--export_quantization_dataset ../../data/c4_demo.json \
--export_size 2 \
--export_legacy_format False

30
examples/qlora_single_gpu/aqlm.sh
View File

@ -1,30 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage sft \
--do_train \
--model_name_or_path BlackSamorez/Llama-2-7b-AQLM-2Bit-1x16-hf \
--dataset alpaca_gpt4_en,glaive_toolcall \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/sft \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 5e-5 \
--num_train_epochs 3.0 \
--max_samples 3000 \
--val_size 0.1 \
--plot_loss \
--fp16

30
examples/qlora_single_gpu/awq.sh
View File

@ -1,30 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage sft \
--do_train \
--model_name_or_path TheBloke/Llama-2-7B-AWQ \
--dataset alpaca_gpt4_en,glaive_toolcall \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/sft \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 5e-5 \
--num_train_epochs 3.0 \
--max_samples 3000 \
--val_size 0.1 \
--plot_loss \
--fp16

31
examples/qlora_single_gpu/bitsandbytes.sh
View File

@ -1,31 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage sft \
--do_train \
--model_name_or_path meta-llama/Llama-2-7b-hf \
--dataset alpaca_gpt4_en,glaive_toolcall \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/sft \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 5e-5 \
--num_train_epochs 3.0 \
--max_samples 3000 \
--val_size 0.1 \
--quantization_bit 4 \
--plot_loss \
--fp16

30
examples/qlora_single_gpu/gptq.sh
View File

@ -1,30 +0,0 @@
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \
--stage sft \
--do_train \
--model_name_or_path TheBloke/Llama-2-7B-GPTQ \
--dataset alpaca_gpt4_en,glaive_toolcall \
--dataset_dir ../../data \
--template default \
--finetuning_type lora \
--lora_target q_proj,v_proj \
--output_dir ../../saves/LLaMA2-7B/lora/sft \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 10 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 5e-5 \
--num_train_epochs 3.0 \
--max_samples 3000 \
--val_size 0.1 \
--plot_loss \
--fp16

27
examples/qlora_single_gpu/llama3_lora_sft_aqlm.yaml Normal file
View File

@ -0,0 +1,27 @@
stage: sft
do_train: true
model_name_or_path: BlackSamorez/Llama-2-7b-AQLM-2Bit-1x16-hf
dataset: alpaca_gpt4_en,glaive_toolcall
dataset_dir: data
template: default
finetuning_type: lora
lora_target: q_proj,v_proj
output_dir: ../../saves/LLaMA2-7B/lora/sft
overwrite_cache: true
overwrite_output_dir: true
cutoff_len: 1024
per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8
lr_scheduler_type: cosine
logging_steps: 10
save_steps: 100
eval_steps: 100
evaluation_strategy: steps
load_best_model_at_end: true
learning_rate: 5e-5
num_train_epochs: 3.0
max_samples: 3000
val_size: 0.1
plot_loss: true
fp16: true

0
examples/qlora_single_gpu/llama3_lora_sft_awq.yaml Normal file
View File

0
examples/qlora_single_gpu/llama3_lora_sft_bitsandbytes.yaml Normal file
View File

0
examples/qlora_single_gpu/llama3_lora_sft_gptq.yaml Normal file
View File

6
setup.py
View File

@ -20,12 +20,12 @@ def get_requires():
extra_require = {
"deepspeed": ["deepspeed>=0.10.0"],
"metrics": ["nltk", "jieba", "rouge-chinese"],
"deepspeed": ["deepspeed>=0.10.0"],
"bitsandbytes": ["bitsandbytes>=0.39.0"],
"vllm": ["vllm>=0.4.0"],
"galore": ["galore-torch"],
"badam": ["badam"],
"vllm": ["vllm>=0.4.0"],
"bitsandbytes": ["bitsandbytes>=0.39.0"],
"gptq": ["optimum>=1.16.0", "auto-gptq>=0.5.0"],
"awq": ["autoawq"],
"aqlm": ["aqlm[gpu]>=1.1.0"],

9
src/webui.py Normal file
View File

@ -0,0 +1,9 @@
from llmtuner.webui.interface import create_ui
def main():
create_ui().queue().launch(server_name="0.0.0.0", server_port=None, share=False)
if __name__ == "__main__":
main()