History

Achazwl c2e086c6ed lora		2022-02-20 17:24:59 +08:00
..
configs	lora	2022-02-20 17:24:59 +08:00
metrics	first commit	2022-02-14 21:19:03 +08:00
README.md	first commit	2022-02-14 21:19:03 +08:00
requirements.txt	first commit	2022-02-14 21:19:03 +08:00
run.sh	first commit	2022-02-14 21:19:03 +08:00
run_glue.py	lora	2022-02-20 17:24:59 +08:00
util.py	first commit	2022-02-14 21:19:03 +08:00

README.md

Text classification with OpenDelta

This repository contains the examples that uses OpenDelta to do text-classification in a traditional classification mode, i.e., with a classification head on top of the language model. Almost all of the training pipeline codes remain the same, except for some minimum changes to insert delta models onto the backbone model.

Generating the json configuration file

python config_gen.py --job $job_name

The available job configuration (e.g., --job lora_roberta-base) can be seen from config_gen.py. You can also create your only configuration.

Run the code

python run_glue.py configs/$job_name/$dataset.json

Possible Errors

ValueError: You must login to the Hugging Face hub on this computer by typing `transformers-cli login` and entering your credentials to use `use_auth_token=Tr
ue`. Alternatively, you can pass your own token as the `use_auth_token` argument.

Solution 1: Please register an account on HuggingFace Then run transformers-cli login on your command line to enter the username and password.
Solution 2: Disable push_to_hub by modifying in the config.json : "push_to_hub": False

OSError: Looks like you do not have git-lfs installed, please install. You can install from https://git-lfs.github.com/. Then run `git lfs install` (you only have to do this once).

Solution 1:

wget -P ~ https://github.com/git-lfs/git-lfs/releases/download/v3.0.2/git-lfs-linux-amd64-v3.0.2.tar.gz
cd ~
tar -xvzf git-lfs-linux-amd64-v3.0.2.tar.gz
export PATH=~:$PATH # a temperary fix. To permantly add, modify your bash
git-lfs install

Solution 2: Disable push_to_hub by modifying in the config.json : "push_to_hub": False

dataset connection error

Solution 1: open a python console, running the error command again, may not be useful

Solution 2: download the dataset by yourself on a internect connected machine, saved to disk and transfer to your server, at last load_from_disk.

Link to the original training scripts

This example repo is based on the huggingface text-classification example. Thanks to the authors of the original repo.