diff --git a/examples/README.md b/examples/README.md index 007a81ab..d5aca5ad 100644 --- a/examples/README.md +++ b/examples/README.md @@ -94,10 +94,10 @@ FORCE_TORCHRUN=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ds3. ### QLoRA Fine-Tuning -#### Supervised Fine-Tuning with 4/8-bit Bitsandbytes Quantization (Recommended) +#### Supervised Fine-Tuning with 4/8-bit Bitsandbytes/HQQ/EETQ Quantization (Recommended) ```bash -llamafactory-cli train examples/train_qlora/llama3_lora_sft_bitsandbytes.yaml +llamafactory-cli train examples/train_qlora/llama3_lora_sft_otfq.yaml ``` #### Supervised Fine-Tuning with 4/8-bit GPTQ Quantization diff --git a/examples/README_zh.md b/examples/README_zh.md index b9d90f25..d96bf882 100644 --- a/examples/README_zh.md +++ b/examples/README_zh.md @@ -94,10 +94,10 @@ FORCE_TORCHRUN=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ds3. ### QLoRA 微调 -#### 基于 4/8 比特 Bitsandbytes 量化进行指令监督微调(推荐) +#### 基于 4/8 比特 Bitsandbytes/HQQ/EETQ 量化进行指令监督微调(推荐) ```bash -llamafactory-cli train examples/train_qlora/llama3_lora_sft_bitsandbytes.yaml +llamafactory-cli train examples/train_qlora/llama3_lora_sft_otfq.yaml ``` #### 基于 4/8 比特 GPTQ 量化进行指令监督微调 diff --git a/examples/train_lora/llama3_lora_dpo.yaml b/examples/train_lora/llama3_lora_dpo.yaml index db25fb51..188e5078 100644 --- a/examples/train_lora/llama3_lora_dpo.yaml +++ b/examples/train_lora/llama3_lora_dpo.yaml @@ -7,7 +7,7 @@ do_train: true finetuning_type: lora lora_target: all pref_beta: 0.1 -pref_loss: sigmoid # [sigmoid (dpo), orpo, simpo] +pref_loss: sigmoid # choices: [sigmoid (dpo), orpo, simpo] ### dataset dataset: dpo_en_demo diff --git a/examples/train_qlora/llama3_lora_sft_bitsandbytes.yaml b/examples/train_qlora/llama3_lora_sft_otfq.yaml similarity index 88% rename from examples/train_qlora/llama3_lora_sft_bitsandbytes.yaml rename to examples/train_qlora/llama3_lora_sft_otfq.yaml index cc773991..9c73b439 100644 --- a/examples/train_qlora/llama3_lora_sft_bitsandbytes.yaml +++ b/examples/train_qlora/llama3_lora_sft_otfq.yaml @@ -1,6 +1,7 @@ ### model model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct quantization_bit: 4 +quantization_method: bitsandbytes # choices: [bitsandbytes (4/8), hqq (2/3/4/5/6/8), eetq (8)] ### method stage: sft