submission_form.txt

### INFO ###
The format for the submission must be a zipfile including:
 - This filled out form
 - Full source code for end-to-end reproducibility
 - Dockerfile-based environment setup
   (we suggest to base your submission on this sandbox repository)
 - Exported .onnx model with batch size = 1
 - Trained .pth checkpoint

### FILL IN ###
# Overall test accuracy reached:
-> 0.562410

# Inference cost score reached:
-> 0.016211

# Complete output dictionary printed by the inference_cost() function:
-> Inference cost for models/model_export.onnx
{
  "discount_sparsity": true,
  "mem_o_FLOAT32": 93208.0,
  "mem_w_SCALEDINT6": 1784.0,
  "op_mac_FLOAT32_SCALEDINT6": 46976.0,
  "op_mac_SCALEDINT8_SCALEDINT6": 18973.0,
  "op_mac_SCALEDUINT6_SCALEDINT6": 258688.0,
  "total_bops": 19242864.0,
  "total_mem_o_bits": 2982656.0,
  "total_mem_w_bits": 10704.0,
  "unsupported": "set()"
}

# Path to .onnx model within this zipfile:
-> ./bacalhaunet_gold_export.onnx

# Path to .pth checkpoint within this zipfile:
-> ./bacalhaunet_gold_weights.pth

# Link to GitHub repository containing your code 
# (to be made public after submission deadline):
-> https://github.com/danielgranhao/ITU-ML5G-PS-007_BacalhauNet_Submission

# Instructions for reproduction:
1. Required python packages must be installed:
   - `pip install odfpy`
2. Environment variables must be set according to `https://github.com/Xilinx/brevitas-radioml-challenge-21` instructions.
3. Execute script `run-docker.sh`
4. Within docker, go to notebooks directory
5. Open a terminal instance and execute `script.sh`
6. When train is complete, the final weights are saved on `models/finalweights.pth`
7. Jupyter notebook `evaluation.ipynb` may be used to confirm accuracy and inference cost score

# Further comments:
In our experience the 56% accuracy requirement may not be met depending on the initial weight initialisation, several executions of the described reproduction procedure may be required.
To overcome this drawback you may load the weights produced after the training procedure and before the 1st pruning:
1. Open `./notebooks/bacalhaunet_quant_reg_prune1.py`
2. Uncomment line 300.
3. Run the commands below:
``` bash
python3 bacalhaunet_quant_reg_prune1.py run_config.ods outputs outputs /workspace/dataset/GOLD_XYZ_OSC.0001_1024.hdf5
python3 bacalhaunet_quant_reg_prune2.py run_config.ods outputs outputs /workspace/dataset/GOLD_XYZ_OSC.0001_1024.hdf5
python3 bacalhaunet_quant_reg_prune3.py run_config.ods outputs outputs /workspace/dataset/GOLD_XYZ_OSC.0001_1024.hdf5
```