Research Area:  Machine Learning

Abstract:

To alleviate the need for large-scale pixel-wise annotations, domain adaptation for semantic segmentation trains segmentation models on synthetic data (source) with computer-generated annotations, which can be then generalized to segment realistic images (target). Recently, self-supervised learning (SSL) with a combination of image-to-image translation shows great effectiveness in adaptive segmentation. The most common practice is to perform SSL along with image translation to well align a single domain (source or target). However, in this single-domain paradigm, unavoidable visual inconsistency raised by image translation may affect subsequent learning. In addition, pseudo labels generated by a single segmentation model aligned in either the source or target domain may be not accurate enough for SSL. In this paper, based on the observation that domain adaptation frameworks performed in the source and target domain are almost complementary, we propose a novel adaptive dual path learning (ADPL) framework to alleviate visual inconsistency and promote pseudo-labeling by introducing two interactive single-domain adaptation paths aligned in source and target domain respectively. To fully explore the potential of this dual-path design, novel technologies such as dual path image translation (DPIT), dual path adaptive segmentation (DPAS), dual path pseudo label generation (DPPLG) and Adaptive ClassMix are proposed. The inference of ADPL is extremely simple, only one segmentation model in the target domain is employed. Our ADPL outperforms the state-of-the-art methods by large margins on GTA5 → Cityscapes, SYNTHIA → Cityscapes and GTA5 → BDD100K scenarios. Code and models are available at https://github.com/royee182/DPL .

Keywords:  
semantic segmentation
Adpl
Adaptive dual path learning

Author(s) Name:  Yiting Cheng,Fangyun Wei,Jianmin Bao,Dong Chen,Wenqiang Zhang

Publisher name:  IEEE

DOI:  10.1109/TPAMI.2023.3248294

Volume Information:  Volume 45,Pages 9339-9356,(2023)

Paper Link:   https://ieeexplore.ieee.org/document/10050808/authors#authors