README.md

👁️VolleyVision👁️

University of Central Asia⛰️

🧪 Example usage

Volleyball	Detection
Action Recognition	Event Detection
Player Detection	Court Detection

🎯 Objectives

✅ Learn and apply popular CV techniques to volleyball data
✅ Popularize volleyball in the field of AI
✅ Create volleyball datasets
✅ Contribute to open-source community
✅ Receive a "thank you" message from anyone who benefited from the project. @volleyfreak ❤️

📝 About

November 7, 2022 | The result of my project should be a web application, that takes a volleyball video and is able to detect and track the ball, players, the court and provides game statistics.

May 15, 2023 | hahahaha, today is the Diploma Project submission day. Well, there is a simple web app, I couldn't deploy it though, and now I am too burned out and busy to get back to this project. hahah I also planned to provide game statistics from the footage, that also did not go as planned. But, overall I am happy)))

🏐 Volleyball Detection & Tracking

Two trained models: RoboFlow (AutoML training) and yoloV7-tiny (local training). Both were trained on my newly created dataset comprised of 25k images. As for the tracker, DaSiamRPN (cv2) was used.

	mAP(50)	precision	recall
yoloV7-tiny	74.1%	86.4%	65.8%
RoboFlow	92.3%	94.7%	86.1%

RoboFlow model is more accurate and works better on official matches, rather than yolov7 model. However, it requires longer time for inference. YoloV7-tiny is capable of real-time inference, even though it is less accurate than RoboFlow model, it is still a good and fast choice for larger volleyballs.

🏃‍♂️ How to Run

• Check out the Stage I - Volleyball/README.md

🏃 Player Detection & 🤾 Action Recognition

Action Recognition

In this stage, I focused on recognizing volleyball actions from the images. A comprehensive volleyball actions dataset was created, comprising 14k images. I used YOLOv8m to train the action recognition model on this dataset.

	mAP50(B)	precision	recall
yolov8m	92.31%	92.38%	89.4%
RoboFlow	83.7%	78.5%	82.3%

The results were highly promising, as shown by the performance metrics.

Players Detection

In addition, I have also trained a YOLOv8m model on a dataset of volleyball players, achieving a high level of accuracy in detecting players in each frame:

	mAP50(B)	precision	recall
yolov8m	97.2%	94.2%	94%
RoboFlow	97.2%	96.7%	91.7%

However, it's worth noting that the yolov8 model, being able to perform in real-time, may occasionally misidentify coaches as players, whereas the RoboFlow model will have a higher rate of false negatives, resulting in missed player detections.

🏃‍♂️ How to Run

• Check out the Stage II - Players & Actions/README.md

🏟️ Court Tracking

In this stage, I trained a simple Semantic Segmentation model on RoboFlow to find and outline the court.

	mIoU
RoboFlow	97.2%

Keep in mind, this was a quick and fun experiment due to an upcoming thesis deadline, and it may not work perfectly every time.

The collage image above should give you the idea about the steps I took to detect the court:

1. Get the segmentation mask from the model
2. Find Contours - cv2.findContours
3. Approximate a polygonal curve - cv2.approxPolyDP

🏃‍♂️ How to Run

• Check out the Stage III - Court Detection/README.md

💾 Datasets

Volleyball	Actions	Players	Spatiotemporal Activity

1. Volleyball (1 class: volleyball)
   • Source Images - 25k_version
   • Duplicates Removed - 19k_version
2. Actions (5 classes: block, defense, serve, set, spike)
   • Source Images - original
   • Resized (640x640) - resized_640
   • Resized (1024x1024) - stretched_dataset
3. Players (1 class: player)
   • Source Images - original
   • Resized (640x640) - resized
   • Augmented Resized (640x640) - aug_res
4. Spatiotemporal Volleyball Activtiy Dataset (7 classes: defense-move, attack, block, reception, service, setting, stand).
   • Source Images - original
   • Resized (640x640) - resized_640

   This dataset belongs to Institute of Computer Graphics and Vision, I downloaded the annotations and the videos from the website, did all the preprocessing and uploaded it to RoboFlow, to make it more accessible and convinient for others to download. Please note that this dataset is video dataset but RoboFlow splitted it randomly into train/test/split. Therefore, after downloading the images and annotations, they all should be combined and then split sequentially.

5. A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions (12 classes: first hit pass, defend, no offensive attack...).
   • HuggingFace - dataset

   MultiSports is a multi-person dataset of spatio-temporal localized sports actions. Please refer to this paper for more details and to this repository for evaluation.

6. Court Segmentation
   • Source Images - original

📝 LICENSE

This project is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives (CC BY-NC-ND) license.

This license allows you to:

• Share — copy and redistribute the material in any medium or format
• Adapt — remix, transform, and build upon the material

Under the following terms:

• Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
• Non-Commercial — You may not use the material for commercial purposes.
• No Derivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.

See the full license text for more details.

Read the full license text

💖 Donate

If you find my work useful or interesting, I would greatly appreciate your support on Ko-fi. Alternatively, you can simply reach out to me and say 'thank you,' which I equally value!)

👨‍💻 Further Development

• Develop and deploy a web app
• Unify all three stages into one coherent and easy to run script
• Research and apply better Court Detection methods
• Train YoloV8 on Volleyball Dataset
• Train 3D Model on Spatiotemporal Activity Dataset
• Extract and transport volleyball actions from MultiSports to RoboFlow
• Combine Players Detection with Actions Recognition to generate personalized statistics
• Combine Volleyball Tracking with Court Detection for smart ball in/out referee system
• Annotate more side-view (official recordings) data for Action Recognition Dataset
• Annotate more data for Court Segmentation Dataset
• Annotate more data for Players Detection Dataset

🐛 Current Drawbacks

• RoboFlow Volleyball Detection Model is slow, due to the API being called on every single frame
• Court Detection is neither robust nor fast
• Action Recognition model is undertrained on Defense class
• Action Recogntiion model is undertrained on side-view data
• Players Detection occasionally detects coaches, referees and spectators as player class

🫶 Contribute

Help me by annoating data, discussing potential improvements or writing a better documentation. Check out the CONTRIBUTING.md. Contributing is easy and appreciated!

🙌 Acknowledgement

• This project wouldn't possible without amazing & free RoboFlow annotation tools , open-source datasets, quick & easy deployement and high-level blog posts. Special thanks to Mohamed Traore for tirelessly approving my constant requests for data storage and model training credits
• Artyom Kulakov - My supervisor. Sincerest gratitude that you foud time to guide and support me along the entire journey (starting from ArtLabs))))
• Nurlan Shaidullaev - FYP course instructor. Thank you for your vigilence and constant reminders about the upcoming deadlines :D
• University of Central Asia - my home for the past 5 years. Immense gratitude to AKDN.

For any additional quesitons feel free to take part in discussions or open an issue.