Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming

https://arxiv.org/pdf/1907.07484.pdf

---

Arixv GPT Summary

The paper "Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming" presents a benchmark to evaluate object detection models' performance under various image corruptions and weather conditions, crucial for real-orld applications like autonomous driving. The authors provide three benchmark datasets with a large variety of image corruptions and show that standard object detection models suffer a severe performance loss on corrupted images, but stylizing the training images leads to a substantial increase in robustness.

Key insights and lessons learned:

- The ability to detect objects in different weather and image conditions is essential for real-world applications like autonomous driving, where safety is a priority.
- Object detection models' performance can degrade significantly under image corruptions and weather conditions that are not present in the training set.
- A simple data augmentation trick, stylizing the training images, can improve object detection models' robustness across different corruption types, severity, and datasets.
- The benchmark datasets provided in the paper can help track future progress towards building more robust object detection models.

Questions for the authors:

- Did you consider other data augmentation techniques besides stylization to improve object detection models' robustness?
- How do you envision using the benchmark datasets to improve the performance of real-world autonomous driving systems?
- Did you test the stylization technique on other computer vision tasks besides object detection?

Future research directions:

Investigating the impact of image corruptions and weather conditions on other computer vision tasks like semantic segmentation or instance segmentation.
Developing new data augmentation techniques that can improve object detection models' robustness to image corruptions and weather conditions.
Evaluating object detection models' robustness in more challenging real-world scenarios like night driving, heavy rain, or snowstorms.

---

논문 "자율 주행에서 겨울이 올 때: 물체 감지의 견고성 벤치마킹"은 다양한 이미지 왜곡과 날씨 조건 하에서 물체 감지 모델의 성능을 평가하는 벤치마크를 제시하며, 이는 자율 주행과 같은 실제 응용 분야에서 중요합니다. 저자들은 다양한 이미지 왜곡을 가진 세 가지 벤치마크 데이터셋을 제공하고, 표준 물체 감지 모델이 왜곡된 이미지에서 심각한 성능 저하를 겪지만, 훈련 이미지의 스타일링은 견고성이 크게 향상되는 것을 보여줍니다.

주요 인사이트 및 배운 교훈:

- 다양한 날씨와 이미지 조건에서 물체를 감지하는 능력은 안전이 우선인 자율 주행과 같은 실제 응용 분야에서 중요합니다.

- 물체 감지 모델의 성능은 훈련 세트에 없는 이미지 왜곡 및 날씨 조건에서 크게 저하될 수 있습니다.

- 훈련 이미지의 스타일링과 같은 간단한 데이터 증강 기술은 다양한 왜곡 유형, 심각도 및 데이터셋에서 물체 감지 모델의 견고성을 향상시킬 수 있습니다.

- 논문에서 제공하는 벤치마크 데이터셋은 더 견고한 물체 감지 모델 구축을 위한 미래 진전 추적에 도움이 될 수 있습니다.

저자들에게 묻는 질문:

- 스타일링 외에도 물체 감지 모델의 견고성을 향상시키기 위해 다른 데이터 증강 기술을 고려했나요?

- 벤치마크 데이터셋을 활용하여 실제 자율 주행 시스템의 성능을 향상시키는 방법을 어떻게 생각하나요?

- 스타일링 기술을 물체 감지 외의 다른 컴퓨터 비전 작업에서도 시험해 보았나요?

미래 연구 방향:

- 시맨틱 분할 또는 인스턴스 분할과 같은 다른 컴퓨터 비전 작업에 대한 이미지 왜곡 및 날씨 조건의 영향을 조사합니다. - 이미지 왜곡 및 날씨 조건에 대한 물체 감지 모델의 견고성을 향상시킬 수 있는 새로운 데이터 증강 기술 개발합니다.

- 야간 운전, heave rain, 또는 눈보라와 같이 보다 어려운 실제 상황에서 물체 감지 모델의 견고성을 평가합니다.

---

 

이미지에 따른 detection 결과

Contribution

1. Object Detection과 instance segmentation 모델들은 corrupted images 에 취약하다는 것을 증명

2. Robust Detection Benchmark 를 제안한다. (PASCAL-C, COCO-C, Cityscape-C) 

3. synthetic corruption 에서 개선된 성능이 자연적 왜곡에 대한 견고성이 증가함을 입증하였다.

4. corruption 에 대한 robustness는 clean data에서도 성능이 향상되었고 간단한 augmentation method는 적은 비용으로 견고성을 향상시키는 방법이다.

5. 여러가지 source 를 공개하였다.

Benchmarking 한 데이터와 데이터 분석 코드

https://github.com/bethgelab/robust-detection-benchmark

GitHub - bethgelab/robust-detection-benchmark: Code, data and benchmark from the paper "Benchmarking Robustness in Object Detect

image corruption library

https://github.com/bethgelab/imagecorruptions

GitHub - bethgelab/imagecorruptions: Python package to corrupt arbitrary images.

https://github.com/bethgelab/stylize-datasets

GitHub - bethgelab/stylize-datasets: A script that applies the AdaIN style transfer method to arbitrary datasets

Examples of image corruption