121 lines
4.1 KiB
Markdown
121 lines
4.1 KiB
Markdown
## Install
|
|
|
|
* Python>=3.8
|
|
|
|
We recommend you to use Anaconda to create a conda environment:
|
|
```bash
|
|
conda create -n MVSOD python=3.8 pip
|
|
```
|
|
Then, activate the environment:
|
|
```bash
|
|
conda activate MVSOD
|
|
```
|
|
|
|
* Other requirements
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
* Build MultiScaleDeformableAttention
|
|
```bash
|
|
cd ./models/ops
|
|
sh ./make.sh # or python setup.py build install
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Dataset preparation
|
|
|
|
1. Please download sky_data3 from [here](https://pan.baidu.com/s/1qI1EjqF5ll7WbzC9H1VjLQ?pwd=1234). The numbers of RGB/IR images in the train, validation and test sets are 8782, 1999, and 1858. The [json](./tools/covert2coco.py) file can be genetated by './tools/covert2coco.py' And the path structure should be as follows:
|
|
|
|
```
|
|
project_root/
|
|
└── data/
|
|
└── vid/
|
|
├── Data
|
|
├── sky_data3/
|
|
└── annotations/
|
|
├── sky_data_vid_test.json
|
|
├── sky_data_vid_train.json
|
|
└── sky_data_vid_val.json
|
|
|
|
```
|
|
|
|
#### Training on multi-gpus
|
|
|
|
```bash
|
|
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
|
|
python tools/launch.py --nnodes 1 --node_rank 0 --master_addr 127.0.0.1 --master_port 3000 --nproc_per_node 2 configs/r101_train_multi_mine_multi.sh
|
|
```
|
|
#### Evaling on multi-gpus
|
|
```
|
|
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
|
|
python tools/launch.py --nnodes 1 --node_rank 0 --master_addr 127.0.0.1 --master_port 3000 --nproc_per_node 2 configs/r101_eval_multi_mine_multi.sh
|
|
```
|
|
1. Pretrained model from [here](https://pan.baidu.com/s/1G10qdtbkbDOZaAoIqiGWXQ?pwd=1234).
|
|
#### Training on single-gpus
|
|
|
|
```bash
|
|
python main.py --backbone resnet101 \
|
|
--epochs 10 \
|
|
--num_feature_levels 1 \
|
|
--num_queries 300 \
|
|
--dilation \
|
|
--batch_size 1 \
|
|
--num_ref_frames 14 \
|
|
--lr_drop_epochs 7 9 \
|
|
--num_workers 8 \
|
|
--with_box_refine \
|
|
--coco_pretrain \
|
|
--dataset_file vid_multi_mine_multi \
|
|
--resume exps/COCO_pretrained_model/checkpoint0020.pth \
|
|
--output_dir exps/singlebaseline/r101_e8_nf4_ld6,7_lr0.0002_nq300_bs4_wbox_joint_MEGA_detrNorm_class31_pretrain_coco_dc5
|
|
|
|
#--resume exps/our_models/COCO_pretrained_model/r101_deformable_detr_single_scale_bbox_refinement-dc5_checkpoint0049.pth \
|
|
#./exps/COCO_pretrained_model/checkpoint0020.pth
|
|
```
|
|
|
|
#### Validating on single-gpus
|
|
|
|
```bash
|
|
python main.py --backbone resnet101 \
|
|
--epochs 6 \
|
|
--eval \
|
|
--num_feature_levels 1 \
|
|
--num_queries 300 \
|
|
--dilation \
|
|
--batch_size 1 \
|
|
--num_ref_frames 14 \
|
|
--resume /home/fty/Documents/fty/mvsod/exps/multibaseline/r101_grad/e7_nf1_ld4,6_lr0.0002_nq300_wbox_MEGA_detrNorm_preSingle_nr14_dc5_nql3_filter150_75_40/checkpoint0012.pth \
|
|
--lr_drop_epochs 4 6 \
|
|
--num_workers 16 \
|
|
--with_box_refine \
|
|
--dataset_file vid_multi_mine_multi \
|
|
--output_dir exps/our_models/exps_multi/r101_81.7
|
|
```
|
|
|
|
### Testing on single-gpus
|
|
|
|
```bash
|
|
python test.py --backbone resnet101 \
|
|
--epochs 7 \
|
|
--eval \
|
|
--num_feature_levels 1 \
|
|
--num_queries 300 \
|
|
--dilation \
|
|
--batch_size 1 \
|
|
--num_ref_frames 14 \
|
|
--resume /home/fty/Documents/fty/mvsod/exps/multibaseline/r101_grad/e7_nf1_ld4,6_lr0.0002_nq300_wbox_MEGA_detrNorm_preSingle_nr14_dc5_nql3_filter150_75_40/checkpoint0099.pth \
|
|
--lr_drop_epochs 4 6 \
|
|
--num_workers 16 \
|
|
--with_box_refine \
|
|
--dataset_file \
|
|
vid_multi_mine_multi \
|
|
--output_dir exps/our_models/exps_multi/epoch100
|
|
```
|
|
The you will get a 'test_save.json' file for evaluation.
|
|
|
|
## Acknowledgment
|
|
This code is based on TransVOD(https://github.com/SJTU-LuHe/TransVOD)
|
|
|