This repo contains the tools needed to create a neural network capable of pose estimation from radar images. Specifically, the programs use the Walabot Creator radar.
- Install the WalabotSDK and Walabot Python library (see python api).
- Install pip dependencies.
- For data collection use
measure.py - For dataset generation use
generate_dataset.py - For data exploration use
explorer.py - For model training use
train.py - For model deployment use
seethru.py
During training models are saved into their seperate subdirectories in the nets folder. Select the wanted model, remove the timestamp from its filename and place it into the nets parent folder. (Like: /nets/seethru.pth)
-
Camera
Ausdom AF640
Resolution: 1080p (running at 640 x 480)
FOV: 90° -
Radar
Vayyar Walabot Creator
Antennas: 15
Board size: 72 mm * 140 mm
Frequency range: 6.3 - 8 GHz
Camera and radar images are stored in png form.
The posed detection model of choice is MediaPipe Pose. Unnecessary keypoints are discarded, so we end up with 13 keypoints. Theese keypoints are the outputs of the model.
A way to improve collected data is to fill in the blank keypoints with ones generated by a nearual network. This is not a hard task, as there is no need to collect new data (altough getting more data always leads to better results). We can process the already collected keypoints by filtering out random keypoints to train a keypoint prediction supplementary neural network. See the implementation of this in supplement_poses.py.
| Measured keypoints | Keypoint supplementation |
|---|---|
 |
 |
To get a neural network that improves over time, we needed to find a loss function that determines how accurate a prediction is. Here N is the number of predicted keypoints, Rx and Ry are vectors of real keypoints and Px and Py are vectors of predicted keypoints.
