42 Commits

Author	SHA1	Message	Date
hiyouga	c9bb0757ec	update pissa example	2024-07-06 15:47:32 +08:00
hiyouga	2f78b5d62a	update examples	2024-06-28 01:17:07 +08:00
hiyouga	095fab58d3	tiny fix about badam	2024-06-25 01:54:53 +08:00
Jonery	97c5235160	add example	2024-06-18 13:50:26 +08:00
hiyouga	2bf2863a58	tiny fix	2024-06-17 17:47:25 +08:00
hiyouga	8c1046d78a	support pissa	2024-06-16 01:08:12 +08:00
hiyouga	b6e008c152	update examples	2024-06-13 03:15:06 +08:00
hiyouga	cae4737907	lora modules: all by default	2024-06-06 03:53:28 +08:00
hiyouga	dc4a00dd63	update train hparams	2024-06-06 01:49:20 +08:00
hiyouga	5a13b3baa6	tiny fix	2024-06-04 00:31:10 +08:00
hiyouga	eed33862bc	fix #4005 #4013	2024-06-03 19:12:29 +08:00
hiyouga	c450ee87a3	improve KTO impl., replace datasets	2024-05-18 03:44:56 +08:00
hiyouga	e5bba7cf1b	update badam example #3764	2024-05-17 02:21:10 +08:00
hiyouga	ddec9e1b84	update examples	2024-05-17 01:02:00 +08:00
hiyouga	2a67ab3925	fix #3694	2024-05-16 00:35:28 +08:00
hiyouga	dae83f4199	update examples	2024-05-13 20:39:36 +08:00
hiyouga	f02f87c6fb	update example docs	2024-05-06 22:51:02 +08:00
hiyouga	34d33e2257	update docs	2024-05-06 21:47:00 +08:00
Oscar	eeb415f6fa	Fix badam example outdated argument	2024-05-05 23:35:19 +08:00
hiyouga	245fe47ece	update webui and add CLIs	2024-05-03 02:58:23 +08:00
hiyouga	a1f1fac33b	update readme and examples	2024-04-22 00:37:32 +08:00
hiyouga	ddbd29d777	remove extras	2024-04-22 00:35:41 +08:00
hiyouga	5c62881c5a	fix bug in galore optimizer	2024-04-21 18:53:22 +08:00
hiyouga	f58425ab45	fix mod stuff	2024-04-21 18:11:10 +08:00
Marco	620add7b9f	Added Mixture of Depths	2024-04-18 20:31:24 +02:00
hoshi-hiyouga	57dcd91e17	Update sft.sh	2024-04-16 17:25:40 +08:00
Jonery	7ecb61822b	resolve gradient checkpointing issue.	2024-04-16 12:05:27 +08:00
Jonery	06c8908d3f	Feature BAdam	2024-04-15 23:15:27 +08:00
hiyouga	cce52351b5	update examples	2024-04-15 22:14:34 +08:00
hiyouga	f22eaeb5bc	update examples	2024-04-02 20:51:21 +08:00
hiyouga	31ffbde24d	update examples	2024-04-02 20:41:49 +08:00
hiyouga	11a6c1bad6	update readme	2024-04-02 20:37:37 +08:00
hiyouga	92dab8a90b	simplify readme	2024-04-02 20:07:43 +08:00
hiyouga	8c77b10912	update trainers	2024-03-28 18:16:27 +08:00
hiyouga	72367307df	improve lora+ impl.	2024-03-13 23:32:51 +08:00
齐保元	a0965cd62c	[FEATURE]: ADD LORA+ ALGORITHM	2024-03-13 19:43:27 +08:00
hiyouga	8664262cde	support layerwise galore	2024-03-10 00:24:11 +08:00
hiyouga	4c00bcdcae	update examples	2024-03-09 02:30:37 +08:00
hiyouga	10be2f0ecc	fix aqlm version	2024-03-09 00:09:09 +08:00
hiyouga	8a45213440	fix example params	2024-03-08 20:41:43 +08:00
hiyouga	33a4c24a8a	fix galore	2024-03-08 00:44:51 +08:00
hiyouga	7230e1177d	add galore examples	2024-03-07 22:53:45 +08:00