Upload

.gitignore

@@ -0,0 +1,166 @@
+__pycache__
+data
+work_dirs
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/

README.md

@@ -0,0 +1,106 @@
+<h2 align="center">ASANet: Asymmetric Semantic Aligning Network for RGB and SAR image land cover classification</h2>
+
+<h5 align="right">by Zhang Pan, Baochai Peng, Chaoran Lu and Quanjin Huang</h5>
+
+
+<div align="center">
+  <img src="https://raw.githubusercontent.com/whu-pzhang/ASANet/master/ASANet_arch.jpg"><br><br>
+</div>
+
+
+This is an official implementation of ASANet in our ISPRS paper [ASANet: Asymmetric Semantic Aligning Network for RGB and SAR image land cover classification](https://www.sciencedirect.com/science/article/abs/pii/S0924271624003630).
+
+[arXiv]()
+
+>Synthetic Aperture Radar (SAR) images have proven to be a valuable cue for multimodal Land Cover Classification (LCC) when combined with RGB images. Most existing studies on cross-modal fusion assume that consistent feature information is necessary between the two modalities, and as a result, they construct networks without adequately addressing the unique characteristics of each modality. In this paper, we propose a novel architecture, named the Asymmetric Semantic Aligning Network (ASANet), which introduces asymmetry at the feature level to address the issue that multi-modal architectures frequently fail to fully utilize complementary features. The core of this network is the Semantic Focusing Module (SFM), which explicitly calculates differential weights for each modality to account for the modality-specific features. Furthermore, ASANet incorporates a Cascade Fusion Module (CFM), which delves deeper into channel and spatial representations to efficiently select features from the two modalities for fusion. Through the collaborative effort of these two modules, the proposed ASANet effectively learns feature correlations between the two modalities and eliminates noise caused by feature differences. Comprehensive experiments demonstrate that ASANet achieves excellent performance on three multimodal datasets. Additionally, we have established a new RGB-SAR multimodal dataset, on which our ASANet outperforms other mainstream methods with improvements ranging from 1.21% to 17.69%. The ASANet runs at 48.7 frames per second (FPS) when the input image is 256 × 256 pixels.
+
+
+
+## Get Started
+
+### install
+
+1. Requirements
+
+* Python 3.8+
+* PyTorch 1.10.0 or higher
+* CUDA 11.1 or higher
+
+
+2. Install all dependencies. Install pytorch, cuda and cudnn, then install other dependencies via:
+
+```
+pip install -r requirements.txt
+```
+
+### Prepare Datasets
+
+1. PIE-RGB-SAR dataset download links [Quark](https://pan.quark.cn/s/383b348cbbea) or [Google Drive](https://drive.google.com/file/d/1O7gNoRTHfxM7ih3CJprvlBijqwYccn2C/view?usp=sharing)
+2. [WHU-RGB-SAR](https://github.com/AmberHen/WHU-OPT-SAR-dataset)
+3. [DDHRNet](https://github.com/XD-MG/DDHRNet/tree/main)
+
+
+The structure of the data file should be like:
+
+```shell
+<datasets>
+|-- <DatasetName1>
+    |-- <RGBFolder>
+        |-- <name1>.<ImageFormat>
+        |-- <name2>.<ImageFormat>
+        ...
+    |-- <SARFolder>
+        |-- <name1>.<ModalXFormat>
+        |-- <name2>.<ModalXFormat>
+        ...
+    |-- <LabelFolder>
+        |-- <name1>.<LabelFormat>
+        |-- <name2>.<LabelFormat>
+        ...
+    |-- train.txt
+    |-- val.txt
+|-- <DatasetName2>
+|-- ...
+```
+
+
+`train.txt` contains the names of items in training set, e.g.:
+
+```shell
+<name1>
+<name2>
+...
+```
+### Training
+
+1. Config
+
+    Edit config file in `configs`, including dataset and network settings.
+
+2. Run multi GPU distributed training:
+
+```shell
+CUDA_VISIBLE_DEVICES="GPU IDs" bash dist_train.sh ${config} ${GPU_NUM} [optional arguments]
+```
+
+### Evaluation
+
+Testing on a single GPU
+
+```shell
+python test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]
+```
+
+## Result
+
+| Model          | Year | FLOPs | Parameter | Speed  | mIoU          |           |               |
+| -------------- | ---- | ----- | --------- | ------ | ------------- | --------- | ------------- |
+|                |      | G     | *M*       | *FPS*  | *PIE-RGB-SAR* | *DDHR-SK* | *WHU-OPT-SAR* |
+| FuseNet        | 2017 | 66    | *55*      | *88.8* | 60.62         | 48.87     | 38.01         |
+| SA-Gate        | 2020 | 46    | 121       | 34.9   | 73.84         | 90.89     | 53.17         |
+| AFNet          | 2021 | 65    | 356       | 35.9   | 76.27         | 91.11     | 53.57         |
+| CMFNet         | 2022 | 77    | 104       | 21.6   | 76.31         | 89.79     | 53.72         |
+| CMX            | 2023 | *15*  | *67*      | 33.5   | *77.10*       | *94.32*   | *55.68*       |
+| FTransUNet     | 2024 | 70    | 203       | 20.7   | 75.72         | 87.64     | 54.47         |
+| *ASANet(ours)* |      | *25*  | 82        | *48.7* | *78.31*       | *94.48*   | *56.11*       |
+

browse_dataset.py

@@ -0,0 +1,82 @@
+import argparse
+import os.path as osp
+
+from mmengine.config import Config, DictAction
+from mmengine.utils import ProgressBar
+
+from mmseg.registry import DATASETS, VISUALIZERS
+from mmseg.utils import register_all_modules
+from mmseg.visualization import SegLocalVisualizer
+
+from src import *
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description='Browse a dataset')
+    parser.add_argument('config', help='train config file path')
+    parser.add_argument(
+        '--output-dir',
+        default=None,
+        type=str,
+        help='If there is no display interface, you can save it')
+    parser.add_argument('--not-show', default=False, action='store_true')
+    parser.add_argument('--show-interval',
+                        type=float,
+                        default=2,
+                        help='the interval of show (s)')
+    parser.add_argument(
+        '--cfg-options',
+        nargs='+',
+        action=DictAction,
+        help='override some settings in the used config, the key-value pair '
+        'in xxx=yyy format will be merged into config file. If the value to '
+        'be overwritten is a list, it should be like key="[a,b]" or key=a,b '
+        'It also allows nested list/tuple values, e.g. key="[(a,b),(c,d)]" '
+        'Note that the quotation marks are necessary and that no white space '
+        'is allowed.')
+    args = parser.parse_args()
+    return args
+
+
+def main():
+    args = parse_args()
+    cfg = Config.fromfile(args.config)
+    if args.cfg_options is not None:
+        cfg.merge_from_dict(args.cfg_options)
+
+    # register all modules in mmdet into the registries
+    register_all_modules()
+
+    dataset = DATASETS.build(cfg.train_dataloader.dataset)
+    cfg.visualizer.update(alpha=0.5)
+    visualizer = VISUALIZERS.build(cfg.visualizer)
+    visualizer.dataset_meta = dataset.metainfo
+
+    progress_bar = ProgressBar(len(dataset))
+    for item in dataset:
+        img = item['inputs'].permute(1, 2, 0).numpy()
+        img1 = img[..., :3]
+        img2 = img[..., 3:]
+        img1 = img1[..., ::-1]  # bgr to rgb
+        img2 = img2[..., ::-1]
+        data_sample = item['data_samples'].numpy()
+        img_path = osp.basename(item['data_samples'].img_path)
+
+        out_file = osp.join(
+            args.output_dir,
+            osp.basename(img_path)) if args.output_dir is not None else None
+
+        visualizer.add_datasample(name=osp.basename(img_path),
+                                  image=img1,
+                                  data_sample=data_sample,
+                                  draw_gt=True,
+                                  draw_pred=False,
+                                  withLabels=False,
+                                  wait_time=args.show_interval,
+                                  out_file=out_file,
+                                  show=not args.not_show)
+        progress_bar.update()
+
+
+if __name__ == '__main__':
+    main()