Merge pull request #25 from lucamarini22/ssl-one-train-set

Ssl one train set

.gitignore

@@ -277,3 +277,5 @@ wandb/*
 config-*.yaml
 conda-*.yaml
 req*.txt
+*.txt
+*.json

README.md

@@ -1,47 +1,88 @@
+<!-- TABLE OF CONTENTS -->
+<details open="open">
+  <summary>Table of Contents</summary>
+  <ol>
+    <li>
+      <a href="#about-the-project">About The Project</a>
+    </li>
+    <li>
+      <a href="#getting-started">Getting Started</a>
+      <ul>
+        <li><a href="#prerequisites">Prerequisites</a></li>
+        <li><a href="#installation">Installation</a></li>
+        <li><a href="#set-up-dataset">Set-up datasets</a></li>
+      </ul>
+    </li>
+    <li><a href="#usage">Usage</a>
+    <ul>
+        <li><a href="#training">Training</a></li>
+        <li><a href="#evaluation">Evaluation</a></li>
+      </ul>
+    </li>
+  </ol>
+</details>
+
+
+
+<!-- ABOUT THE PROJECT -->
 ## About the project
-The anomaly-marine-detection project provides code to apply the state of the art of semi-supervised learning techniques to anomaly detection semantic segmentation problems on satellite imagery of marine regions. The considered anomalies are marine-litter (marine debris), ships, clouds, and algae/organic materials.
+The marine-anomaly-detection project provides code to apply the state of the art of semi-supervised learning techniques to marine anomaly detection semantic segmentation problems on satellite imagery of marine regions. The considered anomalies are marine-litter (marine debris), ships, clouds, and algae/organic materials.
 
 The code builds on and extends the following two repositories:
 - [FixMatch-pytorch](https://github.com/kekmodel/FixMatch-pytorch) implementation based on [PyTorch](https://pytorch.org/). Compared to the original repository, this repository adapts FixMatch to be used for semantic segmentation tasks and to work with multispectral images.
 - [marine-debris.github.io](https://github.com/marine-debris/marine-debris.github.io), which provides the code to work with the [MARIDA](https://marine-debris.github.io/index.html) dataset.
 
+<!-- GETTING STARTED -->
 ## Getting Started
 
 ### Prerequisites
 
-It is recommended to use [conda](https://docs.conda.io/en/latest/) to set-up the environment. [conda](https://docs.conda.io/en/latest/) will take care of all requirements for you. For a detailed list of required packages, please refer to the [conda environment file](https://github.com/lucamarini22/anomaly-marine-detection/blob/main/environment.yml).
+It is recommended to use [conda](https://docs.conda.io/en/latest/) to set-up the environment. [conda](https://docs.conda.io/en/latest/) will take care of all requirements for you. For a detailed list of required packages, please refer to the [conda environment file](https://github.com/lucamarini22/marine-anomaly-detection/blob/main/environment.yml).
 
 ### Installation
 
 1. Get [micromamba](https://mamba.readthedocs.io/en/latest/installation.html#micromamba), or [Miniconda](https://docs.conda.io/en/latest/miniconda.html), or similar. Micromamba is preferred to Miniconda for its greater speed in creating the virtual environment.
 2. Clone the repo.
    ```sh
-   git clone https://github.com/lucamarini22/anomaly-marine-detection.git
+   git clone https://github.com/lucamarini22/marine-anomaly-detection.git
    ```
-3. Setup and activate the environment. This will create a conda environment called `anomaly-marine-detection`.
+3. Setup and activate the environment. This will create a conda environment called `marine-anomaly-detection`.
    ```sh
    micromamba env create -f environment.yml
    ```
    ```sh
-   micromamba activate anomaly-marine-detection
+   micromamba activate marine-anomaly-detection
    ```
 4. [Optional] Install the local package.
    ```
    pip install -e .
    ```
 
+### Set up dataset
+To launch the training on MARIDA, it is necessary to download the dataset. The dataset can be downloaded [here](https://zenodo.org/record/5151941) and has the following structure:
+
+   - `patches`: folder containing the patches (multispectral images).
+   - `splits`: folder containing split files of the training, validation, and test sets.
+
+The `--patches_path` and `--splits_path` arguments in `marineanomalydetection/parse_args_train.py` file shall be respectively point to the `patches` and `splits` folders.
+
+
+<!-- USAGE EXAMPLES -->
+## Usage
+
 ### Training
-1. Create a [Weight and Biases](https://wandb.ai) account to keep track of the experiments.
-2. Set the values of the hyperparameters in this [config.yaml](https://github.com/lucamarini22/anomaly-marine-detection/blob/main/config.yaml) file.
-3. Enter in the main folder
+1. Create a [Weight and Biases](https://wandb.ai) account to keep track of the runs.
+2. Set the values of the hyperparameters in this [config.yaml](https://github.com/lucamarini22/marine-anomaly-detection/blob/main/config.yaml) file.
+3. Enter the main folder.
    ```sh
-   cd /anomaly-marine-detection/
+   cd /marine-anomaly-detection/
    ```
 4. Create a [Sweep](https://docs.wandb.ai/guides/sweeps) to keep track of your training runs.
    ```
-   wandb sweep --project <project-name> config.yaml
+   wandb sweep --project <project-name> <config-file.yaml>
    ```
-5. Start an agent and execute $NUM training runs.
+5. Specify all the values of the arguments in `marineanomalydetection/parse_args_train.py` you did not specify in `<config-file.yaml>`.
+6. Start an agent and execute $NUM training runs.
    ```
    wandb agent --count $NUM <your-entity/sweep-demo-cli/sweepID>
    ```
@@ -51,5 +92,10 @@ It is recommended to use [conda](https://docs.conda.io/en/latest/) to set-up the
    ```
    python evaluation.py --model_path=<path_to_model>
    ```
-2. Visualize the predictions of the last evaluated model by running the cells of the notebook [Visualize Predictions.ipynb](https://github.com/lucamarini22/anomaly-marine-detection/blob/main/notebooks/Visualize%20Predictions.ipynb). Specify the variable `tile_name` to see the predictions for the patches of the specified tile.
+2. Visualize the predictions of the last evaluated model by running the cells of the notebook [Visualize Predictions.ipynb](https://github.com/lucamarini22/marine-anomaly-detection/blob/main/notebooks/Visualize%20Predictions.ipynb). Specify the variable `tile_name` to see the predictions for the patches of the specified tile.
+
+
 
+<!-- ACKNOWLEDGEMENTS 
+This README was based on https://github.com/othneildrew/Best-README-Template
+-->

config.yaml

@@ -9,19 +9,19 @@ parameters:
   lr:
     values: [0.0002, 0.001, 0.01, 0.1]
   threshold:
-    values: [0.6, 0.7, 0.8, 0.9]
+    values: [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
   epochs:
-    value: 900
+    value: 3000
   batch:
-    values: [5, 10]
+    values: [2, 8, 16, 32]
   seed:
     min: 0
     max: 1000
-  mu:
-    values: [5, 9]
+  #mu:
+  #  values: [5, 9]
   reduce_lr_on_plateau:
     values: [0, 1]
   lambda_coeff:
     values: [1] #[0.25, 1]
-  perc_labeled:
-    values: [0.5]
+  #perc_labeled:
+  #  values: [0.5]