DESTRUCTION FROM SKY: WEAKLY SUPERVISED APPROACH FOR DESTRUCTION DETECTION IN SATELLITE IMAGERY

Publication

Muhammad Usman Ali, Waqas Sultani, Mohsen Ali, DESTRUCTION FROM SKY: WEAKLY SUPERVISED APPROACH FOR DESTRUCTION DETECTION IN SATELLITE IMAGERY, ISPRS (2020).

Abstract

Natural and man-made disasters cause huge damage to built infrastructures and results in loss of human lives. The rehabilitation efforts and rescue operations are hampered by the non-availability of accurate and timely information regarding the location of damaged infrastructure and its extent. In this paper, we model the destruction in satellite imagery using a deep learning model employing a weakly-supervised approach. In stark contrast to previous approaches, instead of solving the problem as change detection (using pre and post-event images), we model to identify destruction itself using a single post-event image. To overcome the challenge of collecting pixel-level ground truth data mostly used during training, we only assume image-level labels, representing either destruction is present (at any location) in a given image or not. The proposed attention-based mechanism learns to identify the image-patches with destruction automatically under the sparsity constraint. Furthermore, to reduce false-positive and improve segmentation quality, a hard negative mining technique has been proposed that results in considerable improvement over baseline. To validate our approach, we have collected a new dataset containing destruction and non-destruction images from Indonesia, Yemen, Japan, and Pakistan. On testing-dataset, we obtained excellent destruction results with pixel-level accuracy of 93% and patch level accuracy of 91%. The source code and dataset will be made publicly available.

Contribution

In this paper, we propose to learn a weakly supervised model of destruction just from post-event images, not requiring pre and post-event pair during inference. Our approach is weakly supervised as it needs only image-level labels of destruction and non-destruction and does not require spatially fine-grain (pixel level) or middle level (patch level) annotations for training. To validate our approach, we have collected our destruction dataset which consists of satellite images from different parts of the world such as Yemen, Indonesia, Japan, and Pakistan during different calamities. The main contributions of this work are as follows:

  • We present a weakly-supervised approach to destruction detection by using only weakly labeled satellite imagery. To the best of our knowledge, we are the first to formulate the destruction detection problem in satellite imagery in the context of weakly supervised learning.
  • We propose to use the attention network which facilitates the selection of key destructed patches through the sparsity loss.
  • We introduce a new destruction dataset containing images captures from various parts of the world and cover various sizes and types of destruction. This dataset also includes images without any destruction.
  • Source code and dataset will be shared publicly.

Method

 Architecture of proposed Weakly Supervised Attention Network (WSAN). Given an input training image, we divide it into fixed-size non-overlapping patches and obtain the visual features through DenseNet. For each visual feature (patch) the attention unit generates weights. After multiplying attention weights with their corresponding input feature vector and then taking mean of it, we send it to classifier unit which then classifies the whole large image to either destructed or non-destructed class.

Destruction from Sky Dataset

To get to know the regions where destruction has happened, we used information from different news channels. Images were captured from Google Earth Pro at different zoom levels ranging from 1400ft to 1700ft which means image resolution varies from 40 cm to 80 cm per pixel. Multiple images from different regions are captured at various dates. Images from Sanaa (Yemen) were captured from the years 2015, 2016 and 2018. Images from Palu (Indonesia) belong to the year 2018. Ofunato (Japan) images were taken from the years 2010 and 2011. Similarly, images from Aden (Yemen) were taken from the years 2016 to 2018. Finally, images from Islamabad (Pakistan) were captured from the year 2007. We download satellite images of both destructed and non-destructed areas. We divide the collected dataset into training and testing sets. The training images are of size 672 × 672 and we have 477 images containing destruction and 2253 images containing no destruction at all. The figure below shows some images which represents both destructed and non-destructed regions.

BibTex

@article{ALI2020115,
title = {Destruction from sky: Weakly supervised approach for destruction detection in satellite imagery},
journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
volume = {162},
pages = {115-124},
year = {2020},
issn = {0924-2716},
doi = {https://doi.org/10.1016/j.isprsjprs.2020.02.002},
url = {https://www.sciencedirect.com/science/article/pii/S0924271620300368},
author = {Muhammad Usman Ali and Waqas Sultani and Mohsen Ali},
keywords = {Destruction detection, Segmentation, Attention network, Hard negative mining, Deep learning, Weakly supervised learning}
}