Semantics Sharing

This repository is the offical release for the paper Boosting Real-Time Driving Scene Parsing with Shared Semantics.

Network Architecture

Requirements

• Python 3.6
• PyTorch 1.2
• spatial-correlation-sampler
• kornia

Testing

With the provided pre-trained models, please follow the usages below to perform the testing:

• Ours

  Run the testing script:

  python3 test.py
  

Training

Pre-training

1. For the pre-training of semantic segmentation, please refer to pre_train/semantic_segmentation/README.md.
2. For the pre-training of optical flow, please refer to pre_train/optical_flow/README.md.
3. For the pre-training of feature fusion module ...

Training the whole network

1. Prepare the datasets ...
2. Run the training script ...

License and Citation

All code and other materials (including but not limited to the paper, figures, and tables) are provided for research purposes only and without any warranty. Any commercial use requires our consent. When using any parts of the code package or the paper (Boosting Driving Scene Parsing with Shared Semantics) in your work, please cite the following paper:

@misc{xiang2019boosting,
    title={Boosting Real-Time Driving Scene Parsing with Shared Semantics},
    author={Zhenzhen Xiang and Anbo Bao and Jie Li and Jianbo Su},
    year={2019},
    eprint={1909.07038},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

References

• MobileNetV3:

  @InProceedings{Howard_2019_ICCV,
    author = {Howard, Andrew and Sandler, Mark and Chu, Grace and Chen, Liang-Chieh and Chen, Bo and Tan, Mingxing and Wang, Weijun and Zhu, Yukun and Pang, Ruoming and Vasudevan, Vijay and Le, Quoc V. and Adam, Hartwig},
    title = {Searching for MobileNetV3},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    year = {2019}
    }

• PWC-Net:

  @InProceedings{Sun2018PWC-Net,
    author = {Deqing Sun and Xiaodong Yang and Ming-Yu Liu and Jan Kautz},
    title = {{PWC-Net}: {CNNs} for Optical Flow Using Pyramid, Warping, and Cost Volume},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year = {2018}
    }

Acknowledgments

• Tramac/Lightweight-Segmentation: MobileNetV3 for semantic segmentation.
• RanhaoKang/PWC-Net_pytorch: optical flow estimation.
• ClementPinard/Pytorch-Correlation-extension: PyTorch implementation of Corrleation Module.
• kornia/kornia: PyTorch implementation of homography warping.

Contact

Zhenzhen Xiang (wwfzs1990@gmail.com)