Commit Graph

352 Commits

Author SHA1 Message Date
hiyouga bca1a247bc support lora target auto find 2023-09-09 15:38:37 +08:00
hiyouga d8d82ca281 fix chatglm2 tokenizer 2023-09-09 13:50:29 +08:00
hiyouga d2015c8e12 add baichuan2 convert script 2023-09-08 22:59:41 +08:00
hiyouga 90bd085ae4 fix bug in DPO data collator 2023-09-08 20:45:07 +08:00
hiyouga b34797a845 fix #761 2023-09-08 20:22:18 +08:00
hiyouga 8ea32e4046 change to right-padding, update reward score #803 2023-09-08 20:04:31 +08:00
hiyouga 8aaaa132d4 fix chatglm template 2023-09-08 14:45:58 +08:00
hiyouga 8b66247d2f Update wechat.jpg 2023-09-07 20:56:07 +08:00
hiyouga f5351c18e1 update requirements 2023-09-07 19:26:25 +08:00
hiyouga 5a9970dbef fix #818 2023-09-07 19:19:53 +08:00
hiyouga ed1c2c5557 add deepspeed check in PPO training 2023-09-07 19:12:40 +08:00
hiyouga e2bf7c3bad fix #809 2023-09-07 19:04:32 +08:00
hiyouga 85b1f6632a fix baichuan templates 2023-09-07 18:54:14 +08:00
hiyouga 0531886e1f update baichuan2 template 2023-09-06 21:43:06 +08:00
hiyouga 60603a94c6 add Baichuan2 models 2023-09-06 18:40:11 +08:00
hiyouga 62ce65c628 add Baichuan2 models 2023-09-06 18:36:04 +08:00
hoshi-hiyouga 9224db90ea
Merge pull request #786 from kinghuin/patch-1
fix utils.py bug
2023-09-05 10:49:34 +08:00
Q a19fc2ebf7
fix utils.py bug 2023-09-05 10:38:01 +08:00
hiyouga 370bdb6e43 fix #763 2023-09-01 23:13:05 +08:00
hiyouga a9d1fb72f7 refactor dataset_attr, add eos in pt, fix #757 2023-09-01 19:00:45 +08:00
hiyouga cf106e9d65 Update wechat.jpg 2023-09-01 15:02:23 +08:00
codingma 701a9d60cb
Merge pull request #741 from hiyouga/feature-addDatasetCheck
Feature add dataset check
2023-08-31 20:57:36 +08:00
codemayq 0bcc489c42 update llama2 template 2023-08-30 16:23:56 +08:00
codemayq f7fdc088d4 add dataset stage check 2023-08-30 16:23:08 +08:00
codingma 9b4d16d040
Merge pull request #651 from hiyouga/feature-dataset_stage
add dataset stage
2023-08-28 16:03:45 +08:00
codingma 01dfba85b4
Merge pull request #678 from hiyouga/feature-txt_preview
add text format dataset preview in webui
2023-08-28 16:03:02 +08:00
codemayq 604f85487b add ad gen dataset 2023-08-27 20:35:32 +08:00
hiyouga 15cec650c7 Update wechat.jpg 2023-08-27 01:21:31 +08:00
codemayq 24e68d29f2 add text format dataset preview in webui 2023-08-24 19:45:36 +08:00
codemayq ba94c8729d add stage in DatasetAttr 2023-08-23 20:54:53 +08:00
hiyouga 2de1a7610a fix import error 2023-08-23 20:45:03 +08:00
hiyouga 57146c101f fix #649 2023-08-23 20:21:15 +08:00
codemayq cece66d48a add readme for dataset 2023-08-23 19:55:45 +08:00
codemayq c0e4d1e81b add dataset stage and filter dataset when stage chosen in webui 2023-08-23 18:54:23 +08:00
hiyouga 1c702ad538 fix webui 2023-08-23 11:03:35 +08:00
hoshi-hiyouga c562307476
Merge pull request #644 from hiyouga/fix-quantization_bit
fix quantization bit is ""
2023-08-23 10:45:45 +08:00
codemayq a7cc6c4140 fix quantization bit is "" 2023-08-23 10:08:17 +08:00
codemayq ec2047b064 fix quantization is "" 2023-08-23 10:04:03 +08:00
hiyouga 4318347d3f update template 2023-08-22 19:46:09 +08:00
hoshi-hiyouga 4da719c830
Merge pull request #629 from panpan0000/main
add rm dataset explanation
2023-08-22 13:41:44 +08:00
Peter Pan b0ca8fe634 add rm dataset explanation
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
2023-08-22 01:33:59 -04:00
hoshi-hiyouga bc7795655f
Merge pull request #619 from hiyouga/feature-templateTest
add template encode test
2023-08-21 20:56:34 +08:00
codemayq cbbee7933e add template encode test 2023-08-21 20:51:24 +08:00
hiyouga 5235b15c91 fix #617 2023-08-21 18:16:11 +08:00
hiyouga 02d69b6fde fix #608 2023-08-21 17:49:36 +08:00
hiyouga 0a3f698425 fix baichuan template for training #597 #616 2023-08-21 17:41:51 +08:00
hiyouga 5c052836a0 fix #595 2023-08-20 16:40:00 +08:00
hoshi-hiyouga 1968d9d1d0
Merge pull request #596 from beat4ocean/beat
fix KeyError: 'lang' bug
2023-08-20 16:37:40 +08:00
beat4ocean 7b45de6b9f fix KeyError: 'lang' bug 2023-08-20 15:32:36 +08:00
hiyouga 0676497104 fix ppo trainer #551 2023-08-20 14:07:11 +08:00