This repository demonstrates how to perform hyperparameter tuning using Microsoft’s NNI toolkit. NNI automates feature engineering, neural architecture search, hyperparameter tuning, and model compression for deep learning tasks.
- NNI requires
Python >= 3.7
. It is tested and supported onUbuntu >= 18.04
,Windows 10 >= 21H2
, andmacOS >= 11
- Pip (Python package manager)
-
Open your terminal or command prompt.
-
Clone the repository
git clone https://github.com/mohd-faizy/Hyperparameter-Tuning-with-Microsoft-Network-Intelligence-Toolkit-NNI.git
` -
Navigate to your project directory.
-
Run the following command to create a virtual environment (replace
<environment_name>
with your desired name):python -m venv venv
-
On Windows:
.\venv\Scripts\activate
-
On macOS/Linux:
source ./venv/bin/activate
pip install -r requirements.txt
- Creating the Hyperparameter Search Space:
- In order to perform the hyper-parameter tunning, we first need to create the search space that describs the value range of each hyper-parameter.
- we can use the
.json
code to describe the range & this is the dictionary of all the hyper-parameter values that we want to run for our experiment.
🔸search_space.json
{
"dropout_rate": {
"_type": "uniform",
"_value": [0.1, 0.9]
},
"num_units": {
"_type": "choice",
"_value": [ 32, 64, 128, 256, 512]
},
"lr": {
"_type": "choice",
"_value": [ 0.1, 0.01, 0.001, 0.03, 0.003, 0.06, 0.006]
},
"batch_size": {
"_type": "choice",
"_value": [ 16, 32, 64, 128, 256, 512]
},
"activation": {
"_type": "choice",
"_value": [ "relu", "sigmoid", "tanh"]
}
}
- Then we need to create another file called
config.yml
& which contain all the information regarding the configuration information of our experiment.
🔸config.yml
experimentName: mnist
trialConcurrency: 1
maxExperimentDuration: 1h
maxTrialNumber: 10
searchSpaceFile: search_space.json
useAnnotation: false
trialCommand: python main.py
trialCodeDirectory: .
tuner:
name: TPE
classArgs:
optimize_mode: maximize
trainingService:
platform: local
-
Then at last we need to Execute the following command to start the NNI experiment:
👇
nnictl create --config config.yml
- Open the NNI dashboard in your web browser (usually at http://localhost:8080).
[2024-05-22 00:52:19] Web portal URLs: http://192.168.0.106:8080 http://172.25.208.1:8080 http://127.0.0.1:8080 [2024-05-22 00:52:19] To stop experiment run "nnictl stop c8usp7bz" or "nnictl stop --all" [2024-05-22 00:52:19] Reference: <https://nni.readthedocs.io/en/stable/reference/nnictl.html>
- Observe the trial jobs, intermediate results, and final results.
default metric: 0.9793
Contributions are welcome!
This project is licensed under the MIT License. See LICENSE for details.
If you find this repository helpful, show your support by starring it! For questions or feedback, reach out on Twitter(X
).
🔃 ➤ If you have questions or feedback, feel free to reach out!!!