MLSL: Multi-Level Self-Supervised Learning for Domain Adaptation with Spatially Independent and Semantically Consistent Labeling

 

An illustration of the proposed multi-level self-supervised learning method for domain adaptation of semantic segmentation. (a) Single-inference pseudo-label generation, (b) SISC pseudo-labels generation where, from left to right: patches are extracted randomly, segmented, recombined, normalized and pseudo-labels are generated. (c) shows the semantic segmentation and category-based image classification model, and (d) describes the PWL generation process.

 

Abstract:

Most of the recent Deep Semantic Segmentation algorithms suffer from large generalization errors, even when powerful hierarchical representation models based on convolutional neural networks have been employed.This could be attributed to limited training data and large distribution gap in train and test domain datasets.In this paper, we propose a multi-level self-supervised learning model for domain adaptation of semantic segmentation.Exploiting the idea that an object (and most of the stuff given context) should be labeled consistently regardless of its location, we generate spatially independent and semantically consistent (SISC) pseudo-labels by segmenting multiple sub-images using base model and designing an aggregation strategy. Image level pseudo weak-labels, PWL, are computed to guide domain adaptation by capturing global context similarity in source and domain at latent space level. Thus helping latent space learn the representation even when there are very few pixels belonging to the domain category (small object for example) compared to rest of the image.Our multi-level Self-supervised learning (MLSL) outperforms existing state-of-art (self or adversarial learning) algorithms. Specifically, keeping all setting similar and employing MLSL we obtain an mIoU gain of 5.1% on GTA-V to Cityscapes adaptation and 4.3% on SYNTHIA to Cityscapes adaptation compared to existing state-of-art method.

 

Qualitative Results:


   

Quantitative Results:


 

 

 

Links:

arXiv (MLSL.PDF)

Github ( MLSL)

 

Domain Adaptation for Semantic Segmentation:

Leaderboard( GitHub: Leaderboard: Domain Adaptation for Semantic Segmentation)
You are welcomed to share your published results to be added to leaderboard.

 

Keywords:

Semantic Segmentation, Domain Adaptation, Self-training, Self-supervised learning, Unsupervised Domain Adaptation

 

BIBTEX:

@article{iqbal2019mlsl,
title={MLSL: Multi-Level Self-Supervised Learning for Domain Adaptation with Spatially Independent and Semantically Consistent Labeling},
author={Javed Iqbal and Mohsen Ali},
journal={arXiv preprint arXiv:1909.13776},
year={2019}
}

Contact: javed.iqbal@itu.edu.pk