update readme

This commit is contained in:
hiyouga 2024-03-04 19:29:26 +08:00
parent cda2ff8727
commit 24a79bd50f
3 changed files with 34 additions and 0 deletions

View File

@ -144,6 +144,8 @@ Compared to ChatGLM's [P-Tuning](https://github.com/THUDM/ChatGLM2-6B/tree/main/
Please refer to [constants.py](src/llmtuner/extras/constants.py) for a full list of models we supported.
You also can add a custom chat template to [template.py](src/llmtuner/data/template.py).
## Supported Training Approaches
| Approach | Full-tuning | Freeze-tuning | LoRA | QLoRA |
@ -556,6 +558,8 @@ python src/export_model.py \
> Merging LoRA weights into a quantized model is not supported.
> [!TIP]
> Use `--model_name_or_path path_to_export` only to use the exported model.
>
> Use `--export_quantization_bit 4` and `--export_quantization_dataset data/c4_demo.json` to quantize the model after merging the LoRA weights.
### Inference with OpenAI-style API

View File

@ -144,6 +144,8 @@ https://github.com/hiyouga/LLaMA-Factory/assets/16256802/ec36a9dd-37f4-4f72-81bd
项目所支持模型的完整列表请参阅 [constants.py](src/llmtuner/extras/constants.py)。
您也可以在 [template.py](src/llmtuner/data/template.py) 中添加自己的对话模板。
## 训练方法
| 方法 | 全参数训练 | 部分参数训练 | LoRA | QLoRA |
@ -555,6 +557,8 @@ python src/export_model.py \
> 尚不支持量化模型的 LoRA 权重合并及导出。
> [!TIP]
> 仅使用 `--model_name_or_path path_to_export` 来加载导出后的模型。
>
> 合并 LoRA 权重之后可再次使用 `--export_quantization_bit 4``--export_quantization_dataset data/c4_demo.json` 量化模型。
### 使用 OpenAI 风格 API 推理

View File

@ -213,6 +213,32 @@ def _register_template(
replace_eos: Optional[bool] = False,
force_system: Optional[bool] = False,
) -> None:
r"""
Registers a chat template.
To add the following chat template:
```
[HUMAN]:
user prompt here
[AI]:
model response here
[HUMAN]:
user prompt here
[AI]:
model response here
```
The corresponding code should be:
```
_register_template(
name="custom",
format_user=StringFormatter(slots=["[HUMAN]:\n{{content}}\n[AI]:\n"]),
format_separator=EmptyFormatter(slots=["\n\n"]),
efficient_eos=True,
)
```
"""
eos_slots = [] if efficient_eos else [{"eos_token"}]
template_class = Llama2Template if name.startswith("llama2") else Template
default_user_formatter = StringFormatter(slots=["{{content}}"])