Skip to content

Commit

Permalink
v0.1.11 - Features: added live settings update, mirror option, and im…
Browse files Browse the repository at this point in the history
…proved threading for smoother plugin closure
  • Loading branch information
jellydreams committed Jan 31, 2025
1 parent f1ab8c2 commit b7d0f4b
Show file tree
Hide file tree
Showing 9 changed files with 566 additions and 504 deletions.
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ python app.py
### Build executable

```shell
pyinstaller ./app.py -n VTS_Fullbody_Tracking-0.1.10 --add-data='models/*:models' --add-data="venv/Lib/site-packages/mediapipe/modules/pose_landmark/*:mediapipe/modules/pose_landmark" --add-data="venv/Lib/site-packages/mediapipe/modules/pose_detection/*:mediapipe/modules/pose_detection" --add-data='*.png:.' -F -w
pyinstaller ./app.py -n VTS_Fullbody_Tracking-0.1.11 --add-data='models/*:models' --add-data="venv/Lib/site-packages/mediapipe/modules/pose_landmark/*:mediapipe/modules/pose_landmark" --add-data="venv/Lib/site-packages/mediapipe/modules/pose_detection/*:mediapipe/modules/pose_detection" --add-data='*.png:.' -F -w
```


Expand Down
74 changes: 28 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,75 +1,57 @@
# VTS FULLBODY TRACKING

🔗 website: [vts-fullbody-tracking.gitlab.io](https://vts-fullbody-tracking.gitlab.io/)

[discord-link]: https://discord.gg/9K9gejWQ3s

[![Last Release][github-release-badge]](https://github.com/jellydreams/VTS-Fullbody-Tracking/releases)
[![VTS FULLBODY TRACKING Discord][discord-badge]](https://discord.gg/9K9gejWQ3s)
[![Twitter Follow][twitter-badge]](https://twitter.com/_JellyDreams_)

🔗 website: [vts-fullbody-tracking.gitlab.io](https://vts-fullbody-tracking.gitlab.io/)

![Demo Tracking Arms](readme_img/Demo_Tracking_Arms.png)

[discord-badge]: https://img.shields.io/badge/Join_Discord-indigo?logo=discord&logoColor=white&color=7289da
[twitter-badge]: https://img.shields.io/twitter/follow/_JellyDreams_.svg?style=social
[github-release-badge]: https://img.shields.io/github/v/release/jellydreams/VTS-Fullbody-Tracking?label=ALPHA%20release

This plugin integrates full body tracking functionality using Mediapipe.
It allows users to use tracked body parameters as inputs to control Live2D model in [VTube Studio](https://denchisoft.com/) or [Nizima LIVE](https://nizimalive.com/en/).
Full-body tracking for Live2D VTubers, using a camera.
- **Arms**: Shoulders, Elbow, Wrists
- **Torso**: Clavicles, Hips
- **Legs**: Knees, Ankles
- **Feet**: Heels, Feet Index
- **Hands**: Pinkies Knuckles, Index Knuckles, Thumb Knuckles

![Demo Tracking Arms](readme_img/Demo_Tracking_Arms.png)
Plugin can be used with [VTube Studio](https://denchisoft.com/) or [Nizima LIVE](https://nizimalive.com/en/)

## THIS PLUGIN IS UNDER DEVELOPMENT
This plugin may contain bugs and lack certain features.<br>
Visit the [Wiki](https://github.com/jellydreams/VTS-Fullbody-Tracking/wiki) or [Website](https://vts-fullbody-tracking.gitlab.io/) for more information on how the plugin works. <br>
Join [Discord Server](https://discord.gg/9K9gejWQ3s) to share tests and feedbacks, ask questions or get help.
## Join the Alpha
This plugin can be fixed and improved with feedback.

### How you can Help
- **Live2D Rigger**: Help understand how to effectively rig models for the body parts feature
- **Live2D Vtuber**: Experiment with usability and performance for movement and configuration in VTube Studio or Nizima LIVE
- **Developper**: Contributions are welcome to improve this plugin
Join [Discord Server](https://discord.gg/9K9gejWQ3s) to get help, ask questions, share tests or give feedbacks.<br>

### Troubleshooting
- Currently, there might be a latency of a few seconds
- Tracking input may be inaccurate and exhibit occasional jumps
Visit the [Wiki](https://github.com/jellydreams/VTS-Fullbody-Tracking/wiki) or [Website](https://vts-fullbody-tracking.gitlab.io/) for more information on how the plugin works.

## Run the Plugin

**Requirements**: Window, VTube Studio, Camera
**Requirements**: Window, VTube Studio or Nizima LIVE, Camera

1. Download the executable from the [releases page](https://github.com/jellydreams/VTS-Fullbody-Tracking/releases)
2. **Connect a Camera**
4. **Double-click on the executable file** `VTS_Fullbody_Tracking.exe` to launch the plugin. A settings window will appear.
5. **Select your camera**
3. **Open Software (VTube Studio or Nizima LIVE)**
4. **click on the 'Start Tracking' button**
6. **Allow the plugin in your software**. Window displaying a preview of pose tracking will appear.
7. **Configure your model's parameter settings**, using plugin parameters as inputs. you can now choose body parts X, Y, Z coordinates, and visibility as inputs
1. **Download the executable** from the [releases page](https://github.com/jellydreams/VTS-Fullbody-Tracking/releases)
1. **Connect a camera**
1. **Double-click on the executable file** VTS_Fullbody_Tracking.exe to launch the plugin. A settings window will appear.
1. **Select your camera**. The camera should be different from the one used in Vtube Studio or Nizima LIVE.
1. **Open your software (VTube Studio or Nizima LIVE)** ['Start API' setting should be enabled in VTube Studio]
1. **Click on the 'Start Tracking'** button.
1. **Allow the plugin** in your software. A window displaying a preview of pose tracking will appear.
1. **Configure your model's parameter settings**, using plugin parameters as inputs.

📖 Wiki Section - [Run the plugin](https://github.com/jellydreams/VTS-Fullbody-Tracking/wiki/Run-the-plugin)

## Settings

### Peview Camera
Displays the image captured by the camera

| default | Preview Camera |
|----------------------------------------------------------|----------------------------------------------------------------------|
| ![exemple_preview.png](readme_img/exemple_preview.png) | ![exemple_camera_preview.png](readme_img/exemple_camera_preview.png) |

📖Wiki Section - [Settings Window](https://github.com/jellydreams/VTS-Fullbody-Tracking/wiki/Settings-Window)

## Custom Parameters
This plugin add new controls for various body parts in Vtube Studio. \
Each body part has parameters for controlling its position and visibility.

![List Bodyparts MediaPipe](readme_img/list_bodyparts.png)<br/>
Plugin add new parameters to control live2d model, each body part has parameters for controlling its position (X/Y/Z) and visibility.

- **Arms**: Shoulders, Elbow, Wrists
- **Torso**: Clavicles, Hips
- **Legs**: Knees, Ankles
- **Feet**: Heels, Feet Index
- **Hands**: Pinkies Knuckles, Index Knuckles, Thumb Knuckles
![List Bodyparts MediaPipe](readme_img/list_bodyparts.png)

📖Wiki Section - [List of Parameters](https://github.com/jellydreams/VTS-Fullbody-Tracking/wiki/Custom-Parameters)





Loading

0 comments on commit b7d0f4b

Please sign in to comment.