FCA/README.md

## FCA: Learning a 3D Full-coverage Vehicle Camouflage for Multi-view Physical Adversarial Attack

[AAAI 2022 Paper](https://arxiv.org/abs/2109.07193) FCA: Learning a 3D Full-coverage Vehicle Camouflage for Multi-view Physical Adversarial Attack 

Case study of the FCA. The code can be find in [FCA](https://github.com/idrl-lab/Full-coverage-camouflage-adversarial-attack/tree/gh-pages/src).

### Cases of Digital Attack


#### Carmear distance is 3


<table frame=void>
  <tr>
    <td>before</td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_0_ori_pred.gif?raw=true'/></center></td>
        <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_30_ori_pred.gif?raw=true'/></center></td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_50_ori_pred.gif?raw=true'/></center></td>
  </tr>
    <tr>
    <td>after</td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_0_adv_pred.gif?raw=true'/></center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_30_adv_pred.gif?raw=true'/></center></td>    
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_3_elevation_50_adv_pred.gif?raw=true'/></center></td>
  </tr>
</table>


#### Carmear distance is 5  
<table border=0>
   <tr>
      <td>before</td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_5_elevation_20_ori_pred.gif?raw=true'/></center></td>
         <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_5_elevation_50_ori_pred.gif?raw=true'/></center></td>
  </tr>
    <tr>
     <td>after</td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_5_elevation_20_adv_pred.gif?raw=true'/></center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_5_elevation_50_adv_pred.gif?raw=true'/></center></td>
  </tr>
</table>


#### Carmear distance is 10
<table>
    <tr>
      <td>before</td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_10_elevation_30_ori_pred.gif?raw=true'/></center></td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_10_elevation_50_ori_pred.gif?raw=true'/></center></td>
  </tr>
    <tr>
    <td>after</td>
<td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_10_elevation_30_adv_pred.gif?raw=true'/></center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/distance_10_elevation_50_adv_pred.gif?raw=true'/></center></td>
  </tr>
</table>

### Cases of Multi-view Attack



<table>
  <tr>
  <td>before</td>
  <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_6_ori.png?raw=true'  width="100"/></center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_42_ori.png?raw=true' width="100" /> </center></td>
      <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_54_ori.png?raw=true' width="100" /> </center></td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_126_ori.png?raw=true' width="100" /> </center></td>
      <td><center>  <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_10_330_ori.png?raw=true' width="100" /> </center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_20_78_ori.png?raw=true' width="100" /></center></td>
  </tr>
  <tr>
    <td>after</td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_6_adv.png?raw=true' width="100" /></center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_42_adv.png?raw=true' width="100" /> </center></td>
      <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_54_adv.png?raw=true' width="100" /> </center></td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_0_126_adv.png?raw=true' width="100" /> </center></td>
      <td><center>  <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_10_330_adv.png?raw=true' width="100" /> </center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_1.5_elevation_20_78_adv.png?raw=true' width="100" /></center></td>
  </tr>
</table>

The first row is the original detection result. The second row is the camouflaged detection result.

<table>
  <tr>
   <td>before</td>
  <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_5_elevation_10_57_ori.png?raw=true' width="100" /></center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_5_elevation_30_66_ori.png?raw=true' width="100" /> </center></td>
      <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_10_elevation_0_135_ori.png?raw=true' width="100" /> </center></td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_10_elevation_20_177_ori.png?raw=true' width="100" /> </center></td>
      <td><center>  <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_15_elevation_20_330_ori.png?raw=true' width="100" /> </center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_15_elevation_50_327_ori.png?raw=true' width="100" /></center></td>
  </tr>
  <tr>
    <td>after</td>
  <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_5_elevation_10_57_adv.png?raw=true' width="100" /></center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_5_elevation_30_66_adv.png?raw=true' width="100" /> </center></td>
      <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_10_elevation_0_135_adv.png?raw=true' width="100"/> </center></td>
    <td><center> <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_10_elevation_20_177_adv.png?raw=true' width="100" /> </center></td>
      <td><center>  <img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_15_elevation_20_330_adv.png?raw=true' width="100" /> </center></td>
    <td><center><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/camera_distance_15_elevation_50_327_adv.png?raw=true' width="100"/></center></td>
  </tr>
</table>

The first row is the original detection result. The second row is the camouflaged detection result.

### Ablation study

#### Different combination of loss terms

<img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/abaltion_study_loss.png?raw=true'/>

As we can see from the Figure, different loss terms plays different roles in attacking. For example, the camouflaged car generated by `obj+smooth (we omit the smooth loss, and denotes as obj)` can hidden the vehicle successfully, while the camouflaged car generated by `iou` can successfully suppress the detecting bounding box of the car region, and finally the camouflaged car generated by `cls` successfully make the detector to misclassify the car to anther category.

#### Different initialization ways

<table>
  <tr>
  <td>original </td>
  <td>basic initialization</td>
  <td>random initialization</td>
  <td>zero initialization</td>
  </tr>
  <tr>
  <td><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/data13311_ori.png?raw=true' width="200" /></td>
  <td><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/data13311_adv_basic.png?raw=true' width="200"/></td>
  <td><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/data13311_adv_random.png?raw=true' width="200"/></td>
  <td><img src = 'https://github.com/winterwindwang/Full-coverage-camouflage-adversarial-attack/blob/gh-pages/assets/data13311_adv_zero.png?raw=true' width="200"/></td>
    </tr>
</table>