130 lines
10 KiB
Markdown
130 lines
10 KiB
Markdown
## FCA: Learning a 3D Full-coverage Vehicle Camouflage for Multi-view Physical Adversarial Attack
|
|
|
|
Case study of the FCA. The code can be find in [FCA](https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/tree/gh-pages/src).
|
|
|
|
### Cases of Digital Attack
|
|
|
|
|
|
#### Carmear distance is 3
|
|
|
|
|
|
<table frame=void>
|
|
<tr>
|
|
<td>before</td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_0_ori_pred.gif?raw=true'/></center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_30_ori_pred.gif?raw=true'/></center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_50_ori_pred.gif?raw=true'/></center></td>
|
|
</tr>
|
|
<tr>
|
|
<td>after</td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_0_adv_pred.gif?raw=true'/></center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_30_adv_pred.gif?raw=true'/></center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_50_adv_pred.gif?raw=true'/></center></td>
|
|
</tr>
|
|
</table>
|
|
|
|
|
|
#### Carmear distance is 5
|
|
<table border=0>
|
|
<tr>
|
|
<td>before</td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_5_elevation_20_ori_pred.gif?raw=true'/></center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_5_elevation_50_ori_pred.gif?raw=true'/></center></td>
|
|
</tr>
|
|
<tr>
|
|
<td>after</td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_5_elevation_20_adv_pred.gif?raw=true'/></center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_5_elevation_50_adv_pred.gif?raw=true'/></center></td>
|
|
</tr>
|
|
</table>
|
|
|
|
|
|
#### Carmear distance is 10
|
|
<table>
|
|
<tr>
|
|
<td>before</td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_10_elevation_30_ori_pred.gif?raw=true'/></center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_10_elevation_50_ori_pred.gif?raw=true'/></center></td>
|
|
</tr>
|
|
<tr>
|
|
<td>after</td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_10_elevation_30_adv_pred.gif?raw=true'/></center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_10_elevation_50_adv_pred.gif?raw=true'/></center></td>
|
|
</tr>
|
|
</table>
|
|
|
|
### Cases of Multi-view Attack
|
|
|
|
|
|
|
|
<table>
|
|
<tr>
|
|
<td>before</td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_6_ori.png?raw=true' width="100"/></center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_42_ori.png?raw=true' width="100" /> </center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_54_ori.png?raw=true' width="100" /> </center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_126_ori.png?raw=true' width="100" /> </center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_10_330_ori.png?raw=true' width="100" /> </center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_20_78_ori.png?raw=true' width="100" /></center></td>
|
|
</tr>
|
|
<tr>
|
|
<td>after</td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_6_adv.png?raw=true' width="100" /></center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_42_adv.png?raw=true' width="100" /> </center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_54_adv.png?raw=true' width="100" /> </center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_126_adv.png?raw=true' width="100" /> </center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_10_330_adv.png?raw=true' width="100" /> </center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_20_78_adv.png?raw=true' width="100" /></center></td>
|
|
</tr>
|
|
</table>
|
|
|
|
The first row is the original detection result. The second row is the camouflaged detection result.
|
|
|
|
<table>
|
|
<tr>
|
|
<td>before</td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_5_elevation_10_57_ori.png?raw=true' width="100" /></center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_5_elevation_30_66_ori.png?raw=true' width="100" /> </center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_10_elevation_0_135_ori.png?raw=true' width="100" /> </center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_10_elevation_20_177_ori.png?raw=true' width="100" /> </center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_15_elevation_20_330_ori.png?raw=true' width="100" /> </center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_15_elevation_50_327_ori.png?raw=true' width="100" /></center></td>
|
|
</tr>
|
|
<tr>
|
|
<td>after</td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_5_elevation_10_57_adv.png?raw=true' width="100" /></center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_5_elevation_30_66_adv.png?raw=true' width="100" /> </center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_10_elevation_0_135_adv.png?raw=true' width="100"/> </center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_10_elevation_20_177_adv.png?raw=true' width="100" /> </center></td>
|
|
<td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_15_elevation_20_330_adv.png?raw=true' width="100" /> </center></td>
|
|
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_15_elevation_50_327_adv.png?raw=true' width="100"/></center></td>
|
|
</tr>
|
|
</table>
|
|
|
|
The first row is the original detection result. The second row is the camouflaged detection result.
|
|
|
|
### Ablation study
|
|
|
|
#### Different combination of loss terms
|
|
|
|
<img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/abaltion_study_loss.png?raw=true'/>
|
|
|
|
As we can see from the Figure, different loss terms plays different roles in attacking. For example, the camouflaged car generated by `obj+smooth (we omit the smooth loss, and denotes as obj)` can hidden the vehicle successfully, while the camouflaged car generated by `iou` can successfully suppress the detecting bounding box of the car region, and finally the camouflaged car generated by `cls` successfully make the detector to misclassify the car to anther category.
|
|
|
|
#### Different initialization ways
|
|
|
|
<table>
|
|
<tr>
|
|
<td>original </td>
|
|
<td>basic initialization</td>
|
|
<td>random initialization</td>
|
|
<td>zero initialization</td>
|
|
</tr>
|
|
<tr>
|
|
<td><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/data13311_ori.png?raw=true' width="200" /></td>
|
|
<td><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/data13311_adv_basic.png?raw=true' width="200"/></td>
|
|
<td><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/data13311_adv_random.png?raw=true' width="200"/></td>
|
|
<td><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/data13311_adv_zero.png?raw=true' width="200"/></td>
|
|
</tr>
|
|
</table>
|