History

shengdinghu c161440867 merge parallel adapter		2022-10-17 06:37:17 +00:00
..
backbones	resolve conflict	2022-07-29 23:01:08 +08:00
configs	merge parallel adapter	2022-10-17 06:37:17 +00:00
data_processors	resolve conflict	2022-07-29 23:01:08 +08:00
hyperopt	update examples	2022-04-22 19:18:25 +08:00
metrics	backbones and corresponding configs	2022-07-29 21:26:51 +08:00
sfs_scripts	search	2022-04-14 11:22:41 +08:00
src	resolve conflict	2022-07-29 23:01:08 +08:00
utils	resolve conflict	2022-07-29 23:01:08 +08:00
README.md	finish delta center development	2022-07-03 02:10:18 +00:00
__init__.py	add example	2022-02-26 14:24:42 +08:00
requirements.txt	fixbug	2022-06-06 16:21:55 +08:00

README.md

Examples of using opendelta together with 🤗 transformers.

In this repo, we construct a very general pipeline to train and test a PLM using 🤗 transformers.

The pipeline was constructed together with openpromptu, which is a light and model-agnostic version of openprompt.

Pool of PLMs

We are going to adapt most of the models in 🤗 transformers in the repos. The different pipeline, processing, or configurations are specified in ./backbones/. You can add your own model in this file to support customized models.

A example script to run the repo in offline mode

conda activate [YOURENV]
PATHBASE=[YOURPATH]

JOBNAME="adapter_t5-base"
DATASET="superglue-cb"

cd $PATHBASE/OpenDelta/examples/examples_prompt/
python configs/gen_t5.py --job $JOBNAME

export TRANSFORMERS_OFFLINE=1
export HF_DATASETS_OFFLINE=1
python src/run.py configs/$JOBNAME/$DATASET.json \
--model_name_or_path [YOURPATH_TO_T5_BASE] \
--tokenizer_name [YOURPATH_TO_T5_BASE] \
--datasets_saved_path [YOURPATH_TO_CB_DATASETS] \
--finetuned_delta_path ${PATHBASE}/delta_checkpoints/ \
--num_train_epochs 20 \
--bottleneck_dim 24 \
--delay_push True

A example of quick testing the repo.

conda activate [YOURENV]
PATHBASE=[YOURPATH]

JOBNAME="adapter_t5-base"
DATASET="superglue-cb"

cd $PATHBASE/OpenDelta/examples/examples_prompt/

export TRANSFORMERS_OFFLINE=1
export HF_DATASETS_OFFLINE=1
export DELTACENTER_OFFLINE=0
python src/test.py configs/$JOBNAME/$DATASET.json \
--model_name_or_path [YOURPATH_TO_T5_BASE] \
--tokenizer_name [YOURPATH_TO_T5_BASE] \
--datasets_saved_path [YOURPATH_TO_CB_DATASETS] \
--finetuned_delta_path thunlp/t5-base_adapter_superglue-cb_20220701171436c80 \
--delta_cache_dir "./delta_checkpoints/" \
--force_download True