hiyouga
|
b240b6792f
|
fix webui
|
2023-10-13 16:27:59 +08:00 |
|
hiyouga
|
11bd271364
|
fix ppo args
|
2023-10-11 23:40:50 +08:00 |
|
hiyouga
|
2818af0b09
|
refactor model_dtype, fix PPO trainer
|
2023-10-11 23:16:01 +08:00 |
|
hiyouga
|
5310e4d182
|
add averaging in evaluation
|
2023-10-10 23:16:31 +08:00 |
|
hiyouga
|
be420e4179
|
fix aquila template, repair sft packing mechanism
|
2023-10-10 18:49:55 +08:00 |
|
hiyouga
|
e1dcb8e4dc
|
tiny fix
|
2023-10-10 17:41:13 +08:00 |
|
hiyouga
|
0a356bc897
|
fix flash shift short attention
|
2023-10-09 17:54:48 +08:00 |
|
hiyouga
|
6b24f29c8a
|
fix webui args
|
2023-10-09 17:13:57 +08:00 |
|
hiyouga
|
ab65c3063b
|
fix shift short attention
|
2023-10-09 17:07:46 +08:00 |
|
hiyouga
|
b8dbec086e
|
update webui #1086
|
2023-10-09 14:50:14 +08:00 |
|
hiyouga
|
a683c5b797
|
fix #1097
|
2023-10-08 22:29:26 +08:00 |
|
hiyouga
|
f9769cff8a
|
add llamafy_qwen.py
|
2023-10-08 22:05:36 +08:00 |
|
hiyouga
|
d11a545463
|
fix #1068 #1074
|
2023-09-28 14:39:16 +08:00 |
|
hiyouga
|
de19614306
|
fix bug in packed sft dataset
|
2023-09-28 01:16:46 +08:00 |
|
hiyouga
|
5d4118b096
|
tiny fix
|
2023-09-28 01:03:04 +08:00 |
|
hiyouga
|
d2ebd225db
|
tiny fix
|
2023-09-28 01:02:11 +08:00 |
|
hiyouga
|
c902236397
|
fix #1064
|
2023-09-28 00:53:29 +08:00 |
|
hiyouga
|
b3fbba57eb
|
fix bug in pretraining
|
2023-09-28 00:45:20 +08:00 |
|
hiyouga
|
84b7486885
|
fix layer norm dtype
|
2023-09-28 00:25:55 +08:00 |
|
hiyouga
|
b0b0138e1d
|
fix #1026
|
2023-09-27 22:57:09 +08:00 |
|
hiyouga
|
35fa94723c
|
fix #424
|
2023-09-27 22:49:43 +08:00 |
|
hiyouga
|
a41e12de5e
|
fix #1032
|
2023-09-27 22:42:16 +08:00 |
|
hiyouga
|
620efe1d8d
|
refactor finetuning Args
|
2023-09-27 22:28:06 +08:00 |
|
hiyouga
|
90375f600d
|
support LongLoRA
|
2023-09-27 21:55:50 +08:00 |
|
hiyouga
|
4dd9b4d982
|
add CMMLU, update eval script
|
2023-09-23 21:10:17 +08:00 |
|
hiyouga
|
f8ff625d76
|
update evaluate
|
2023-09-23 11:55:31 +08:00 |
|
hiyouga
|
badd2735b5
|
move file
|
2023-09-23 11:52:12 +08:00 |
|
hiyouga
|
465ee8119a
|
add MMLU and C-Eval script
|
2023-09-23 00:34:17 +08:00 |
|
hiyouga
|
5cc7a44784
|
fix #1000
|
2023-09-22 15:00:48 +08:00 |
|
hiyouga
|
5f3ab3ddde
|
fix webui
|
2023-09-21 19:55:38 +08:00 |
|
hiyouga
|
dbaef776a1
|
tiny fix
|
2023-09-21 19:52:06 +08:00 |
|
hiyouga
|
338b8664ed
|
fix #944
|
2023-09-21 19:51:02 +08:00 |
|
hiyouga
|
ace3f85a72
|
tiny fix
|
2023-09-21 15:25:29 +08:00 |
|
statelesshz
|
b3e41c6d49
|
support export model on Ascend NPU
|
2023-09-20 10:26:02 +08:00 |
|
hiyouga
|
10ab2f8b90
|
fix webui
|
2023-09-19 18:35:21 +08:00 |
|
hiyouga
|
7e8655c8b5
|
fix error info
|
2023-09-19 18:30:23 +08:00 |
|
hiyouga
|
469f859161
|
add tests.cal_flops.py
|
2023-09-16 23:40:41 +08:00 |
|
hiyouga
|
0b5f970c05
|
fix #913
|
2023-09-15 20:58:28 +08:00 |
|
hiyouga
|
8632bff811
|
fix #896
|
2023-09-14 18:37:34 +08:00 |
|
hiyouga
|
8857e45602
|
fix #887
|
2023-09-14 17:56:58 +08:00 |
|
mmbwf
|
30fb721f12
|
Update utils.py
Fix parameters load error.
|
2023-09-14 15:38:04 +08:00 |
|
hiyouga
|
7ba57d5b14
|
fix ppo save model
|
2023-09-12 16:25:29 +08:00 |
|
hiyouga
|
d4be857e23
|
fix #762 #814
|
2023-09-12 16:10:10 +08:00 |
|
hiyouga
|
3b306478d4
|
tiny fix
|
2023-09-11 18:27:08 +08:00 |
|
hiyouga
|
ccb3553576
|
Release v0.1.8
|
2023-09-11 17:31:34 +08:00 |
|
hiyouga
|
0fbece85a7
|
update flashattn, fix ppo save model
|
2023-09-11 17:25:36 +08:00 |
|
hiyouga
|
b218c271ed
|
remove PeftTrainer
|
2023-09-10 22:23:23 +08:00 |
|
hiyouga
|
d8aa1404be
|
support FlashAttention2
|
2023-09-10 20:43:56 +08:00 |
|
hiyouga
|
815b92e698
|
fix #850
|
2023-09-10 14:22:03 +08:00 |
|
hiyouga
|
a51b7c98ac
|
fix lora target
|
2023-09-09 17:04:45 +08:00 |
|
hiyouga
|
bca1a247bc
|
support lora target auto find
|
2023-09-09 15:38:37 +08:00 |
|
hiyouga
|
d8d82ca281
|
fix chatglm2 tokenizer
|
2023-09-09 13:50:29 +08:00 |
|
hiyouga
|
90bd085ae4
|
fix bug in DPO data collator
|
2023-09-08 20:45:07 +08:00 |
|
hiyouga
|
b34797a845
|
fix #761
|
2023-09-08 20:22:18 +08:00 |
|
hiyouga
|
8ea32e4046
|
change to right-padding, update reward score #803
|
2023-09-08 20:04:31 +08:00 |
|
hiyouga
|
8aaaa132d4
|
fix chatglm template
|
2023-09-08 14:45:58 +08:00 |
|
hiyouga
|
f5351c18e1
|
update requirements
|
2023-09-07 19:26:25 +08:00 |
|
hiyouga
|
5a9970dbef
|
fix #818
|
2023-09-07 19:19:53 +08:00 |
|
hiyouga
|
ed1c2c5557
|
add deepspeed check in PPO training
|
2023-09-07 19:12:40 +08:00 |
|
hiyouga
|
e2bf7c3bad
|
fix #809
|
2023-09-07 19:04:32 +08:00 |
|
hiyouga
|
85b1f6632a
|
fix baichuan templates
|
2023-09-07 18:54:14 +08:00 |
|
hiyouga
|
0531886e1f
|
update baichuan2 template
|
2023-09-06 21:43:06 +08:00 |
|
hiyouga
|
62ce65c628
|
add Baichuan2 models
|
2023-09-06 18:36:04 +08:00 |
|
Q
|
a19fc2ebf7
|
fix utils.py bug
|
2023-09-05 10:38:01 +08:00 |
|
hiyouga
|
370bdb6e43
|
fix #763
|
2023-09-01 23:13:05 +08:00 |
|
hiyouga
|
a9d1fb72f7
|
refactor dataset_attr, add eos in pt, fix #757
|
2023-09-01 19:00:45 +08:00 |
|
codemayq
|
0bcc489c42
|
update llama2 template
|
2023-08-30 16:23:56 +08:00 |
|
codemayq
|
f7fdc088d4
|
add dataset stage check
|
2023-08-30 16:23:08 +08:00 |
|
codingma
|
9b4d16d040
|
Merge pull request #651 from hiyouga/feature-dataset_stage
add dataset stage
|
2023-08-28 16:03:45 +08:00 |
|
codemayq
|
24e68d29f2
|
add text format dataset preview in webui
|
2023-08-24 19:45:36 +08:00 |
|
codemayq
|
ba94c8729d
|
add stage in DatasetAttr
|
2023-08-23 20:54:53 +08:00 |
|
hiyouga
|
2de1a7610a
|
fix import error
|
2023-08-23 20:45:03 +08:00 |
|
hiyouga
|
57146c101f
|
fix #649
|
2023-08-23 20:21:15 +08:00 |
|
codemayq
|
c0e4d1e81b
|
add dataset stage and filter dataset when stage chosen in webui
|
2023-08-23 18:54:23 +08:00 |
|
hiyouga
|
1c702ad538
|
fix webui
|
2023-08-23 11:03:35 +08:00 |
|
codemayq
|
a7cc6c4140
|
fix quantization bit is ""
|
2023-08-23 10:08:17 +08:00 |
|
codemayq
|
ec2047b064
|
fix quantization is ""
|
2023-08-23 10:04:03 +08:00 |
|
hiyouga
|
4318347d3f
|
update template
|
2023-08-22 19:46:09 +08:00 |
|
hiyouga
|
5235b15c91
|
fix #617
|
2023-08-21 18:16:11 +08:00 |
|
hiyouga
|
02d69b6fde
|
fix #608
|
2023-08-21 17:49:36 +08:00 |
|
hiyouga
|
0a3f698425
|
fix baichuan template for training #597 #616
|
2023-08-21 17:41:51 +08:00 |
|
hiyouga
|
5c052836a0
|
fix #595
|
2023-08-20 16:40:00 +08:00 |
|
beat4ocean
|
7b45de6b9f
|
fix KeyError: 'lang' bug
|
2023-08-20 15:32:36 +08:00 |
|
hiyouga
|
0676497104
|
fix ppo trainer #551
|
2023-08-20 14:07:11 +08:00 |
|
hiyouga
|
9c9009f49f
|
Release v0.1.7
|
2023-08-18 17:21:27 +08:00 |
|
hiyouga
|
d75e377b0f
|
tiny fix
|
2023-08-18 13:07:35 +08:00 |
|
hiyouga
|
53e33418d0
|
support ppo score norm (trl 0.5.1.dev required)
|
2023-08-18 12:02:42 +08:00 |
|
hiyouga
|
9020524418
|
fix PPO trainer #551 , update readme
|
2023-08-18 11:43:10 +08:00 |
|
hiyouga
|
58f13e22da
|
update training resuming
|
2023-08-18 01:41:17 +08:00 |
|
hoshi-hiyouga
|
7252903245
|
Merge branch 'main' into main
|
2023-08-18 01:37:23 +08:00 |
|
hiyouga
|
d125218cde
|
support bf16 ppo #551
|
2023-08-18 00:40:32 +08:00 |
|
hiyouga
|
9f4c2adc9a
|
fix ChatGLM2 ppo #527 #528
|
2023-08-18 00:34:59 +08:00 |
|
hiyouga
|
be21fc83f9
|
fix generation bug #532
|
2023-08-17 22:21:34 +08:00 |
|
hiyouga
|
b0ed0dec5e
|
fix streaming in pt stage #548 #549
|
2023-08-17 17:59:26 +08:00 |
|
hiyouga
|
892fd39373
|
fix baichuan and intern template
|
2023-08-17 01:27:20 +08:00 |
|
hiyouga
|
d9e62711a3
|
fix generation
|
2023-08-16 22:39:54 +08:00 |
|
hiyouga
|
7407d9daa1
|
fix system prompt
|
2023-08-16 01:35:52 +08:00 |
|
hiyouga
|
273135f595
|
fix baichuan template #481
|
2023-08-15 11:38:21 +08:00 |
|
hiyouga
|
af6c011fcb
|
fix ChatGLM RLHF
|
2023-08-15 11:19:20 +08:00 |
|
hiyouga
|
80b4053602
|
alert pad_token source
|
2023-08-15 00:07:56 +08:00 |
|
hiyouga
|
9d0f6214b6
|
update webui
|
2023-08-14 22:45:26 +08:00 |
|
hoshi-hiyouga
|
adb0f186e9
|
Merge pull request #511 from hiyouga/feature-autoTemplate
add template match and stage in webui
|
2023-08-14 22:44:04 +08:00 |
|
codemayq
|
0bf892ff1a
|
auto match template when change model_name
|
2023-08-14 20:56:05 +08:00 |
|
codemayq
|
79c68e5527
|
add template match and stage in webui
|
2023-08-14 20:42:59 +08:00 |
|
hiyouga
|
d019956808
|
fix ChatGLM lm_head #494
|
2023-08-14 14:14:48 +08:00 |
|
hiyouga
|
20a29297b1
|
fix bug in webui
|
2023-08-14 11:38:42 +08:00 |
|
hiyouga
|
ca08e5efd3
|
fix webui cache
|
2023-08-14 11:37:01 +08:00 |
|
hiyouga
|
ec94274ca1
|
web UI integrating RLHF
|
2023-08-14 10:48:47 +08:00 |
|
hiyouga
|
2f2fd55d81
|
fix #480
|
2023-08-14 00:23:56 +08:00 |
|
hiyouga
|
d69b1388e6
|
fix webui
|
2023-08-12 23:52:07 +08:00 |
|
hiyouga
|
9dc6a296e3
|
tiny fix
|
2023-08-12 22:02:43 +08:00 |
|
hiyouga
|
8545c11c45
|
fix rope scaling
|
2023-08-12 22:00:01 +08:00 |
|
hiyouga
|
1836c020c5
|
update readme
|
2023-08-12 21:00:11 +08:00 |
|
hiyouga
|
fa940c17b8
|
support rope scaling, fix #475 #476 #478
|
2023-08-12 20:46:27 +08:00 |
|
codemayq
|
6bc8e9866d
|
add sft script preview in webui
|
2023-08-12 13:53:55 +08:00 |
|
hiyouga
|
dd51c24203
|
fix unusual output of 8bit models #278 #391
|
2023-08-12 00:25:29 +08:00 |
|
hiyouga
|
a48cb0d474
|
Release v0.1.6
|
2023-08-11 23:25:57 +08:00 |
|
hiyouga
|
d3844e97e3
|
add defaults
|
2023-08-11 13:56:26 +08:00 |
|
hiyouga
|
d59f938959
|
fix stop word in baichuan template
|
2023-08-11 13:51:46 +08:00 |
|
hiyouga
|
9c6dd10514
|
fix baichuan template
|
2023-08-11 13:45:47 +08:00 |
|
hiyouga
|
3ec4351cfd
|
support DPO training (2305.18290)
|
2023-08-11 03:02:53 +08:00 |
|
hoshi-hiyouga
|
685dae4eff
|
Merge pull request #451 from jovialchen/main
huggingface login for projects must login while running
|
2023-08-10 17:25:38 +08:00 |
|
hiyouga
|
ad6e7c76c7
|
fix webui val size
|
2023-08-10 15:20:44 +08:00 |
|
jiongxuc
|
3e000c2b60
|
huggingface login for projects must login while running
|
2023-08-10 14:57:12 +08:00 |
|
hiyouga
|
eb6e571cb7
|
fix template
|
2023-08-09 23:14:27 +08:00 |
|
hiyouga
|
ac29f4d5f0
|
fix template
|
2023-08-09 23:10:20 +08:00 |
|
hiyouga
|
d86ea314a1
|
support val set in streaming mode
|
2023-08-09 23:00:26 +08:00 |
|
hiyouga
|
572ea3bafb
|
fix tokenizer
|
2023-08-09 17:52:15 +08:00 |
|
niuba
|
2ec68d3398
|
add last_checkpoint support
|
2023-08-09 16:39:27 +08:00 |
|
hiyouga
|
df946e6949
|
fix sft trainer
|
2023-08-09 16:35:03 +08:00 |
|
hiyouga
|
39cd8b6989
|
fix rm #420, fix template #426, fix #423
|
2023-08-09 16:23:31 +08:00 |
|
hoshi-hiyouga
|
2d90685358
|
fix llama2 template
|
2023-08-09 00:58:27 +08:00 |
|
hoshi-hiyouga
|
32fa5e8d70
|
fix tokenizer
|
2023-08-09 00:54:54 +08:00 |
|
hiyouga
|
3a720aac66
|
update webui
|
2023-08-09 00:26:11 +08:00 |
|
hiyouga
|
eecc4b2131
|
fix tokenizer #417
|
2023-08-08 23:59:41 +08:00 |
|
hiyouga
|
caa0eda27d
|
fix bug
|
2023-08-08 21:28:28 +08:00 |
|
hiyouga
|
4b841a6b35
|
fix bug
|
2023-08-08 17:55:55 +08:00 |
|
hiyouga
|
a9980617f5
|
fix chatml template #408
|
2023-08-08 17:44:39 +08:00 |
|
hiyouga
|
5453b93db0
|
update args spec
|
2023-08-07 15:23:35 +08:00 |
|
hiyouga
|
20cf27976f
|
update readme
|
2023-08-07 15:02:02 +08:00 |
|
hiyouga
|
081345baca
|
fix #376
|
2023-08-07 13:58:59 +08:00 |
|
hiyouga
|
220175ab24
|
update trainer
|
2023-08-07 13:34:35 +08:00 |
|
hiyouga
|
e21ae01356
|
fix qwen eos token
|
2023-08-06 13:31:17 +08:00 |
|
hiyouga
|
7f18d2a335
|
fix qwen tokenizer #361
|
2023-08-05 17:06:05 +08:00 |
|
hiyouga
|
1afa51c2fa
|
fix template for tiktoken
|
2023-08-05 13:42:42 +08:00 |
|
hiyouga
|
53d95725c5
|
remove redundant code
|
2023-08-05 00:27:27 +08:00 |
|
hiyouga
|
c183b3551d
|
fix template
|
2023-08-05 00:25:00 +08:00 |
|
hiyouga
|
e4a15f863c
|
fix llama2 template
|
2023-08-05 00:07:54 +08:00 |
|
hiyouga
|
d87c8fd8ab
|
fix bos and eos token
|
2023-08-04 23:55:57 +08:00 |
|
hiyouga
|
8172ad1b5e
|
fix encode
|
2023-08-04 23:27:55 +08:00 |