Skip to content

This project integrates MediaPipe Solutions with Node.js and Express for real-time computer vision tasks. It showcases examples of image segmentation, hand and face detection, and pose detection, with a combined example for all three types of landmark detection.

License

Apache-2.0, Apache-2.0 licenses found

Licenses found

Apache-2.0
LICENSE
Apache-2.0
LICENSE_Mediapipe
Notifications You must be signed in to change notification settings

heyfoz/nodejs-mediapipe

mediapipe-node

Hand-Face Example

This project integrates MediaPipe Solutions with Node.js and Express for real-time computer vision tasks. It showcases examples of image segmentation, hand and face detection, and pose detection, with a combined example for all three types of landmark detection.

Utilizing MediaPipe's client-side JavaScript APIs, the project performs video processing directly in the browser, minimizing server-side computation. Additionally, it features logging for hand and face gestures, tracking interactions both in the console and a text file in the server logs directory.

Project Structure

mediapipe-node/
├── server.js
├── package.json
├── package-lock.json
├── public/
│   ├── css/
│   │   ├── full_detection_styles.css
│   │   ├── hand_face_detection_styles.css
│   │   ├── image_segmentation_styles.css
│   │   ├── main.css
│   │   └── pose_styles.css
│   ├── images/
│   │   ├── face_landmark.png
│   │   ├── face_landmarker_keypoints.png
│   │   ├── hand-landmarks.png
│   │   ├── hand_landmark.png
│   │   ├── pose_detector.png
│   │   └── pose_landmarks_index.png
│   ├── js/
│   │   ├── full_detection.js
│   │   ├── hand_face_detection.js
│   │   ├── image_segmentation.js
│   │   └── pose_detection.js
│   └── json/
│       ├── blendshape_map.json
│       ├── hand_map.json
│       ├── gesture_map.json
│       └── pose_map.json
├── models/
│   ├── hand_landmarker.task
│   ├── face_landmarker.task
│   ├── pose_landmarker_full.task
│   └── selfie_multiclass_256x256.tflite
└── templates/
    ├── full_detection.html
    ├── hand_face_detection.html
    ├── image_segmentation.html
    ├── index.html
    └── pose_detection.html

Setup Instructions

  1. Install Node.js

    Follow the instructions on the Node.js official website to download and install Node.js. This will also install npm (Node Package Manager), which is required to manage project dependencies.

  2. Clone the Repository

    git clone https://github.com/heyfoz/nodejs-mediapipe.git
    cd mediapipe-node
  3. Install Dependencies

    Install the necessary Node.js packages:

    npm install express
    npm install express-validator
  4. Run the Application

    Start the server by running:

    node server.js
  5. Access the Application

    Open your browser and navigate to http://localhost:3000 to view the homepage.

Project License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.

MediaPipe License

The MediaPipe models and code included in this project are licensed under the Apache License, Version 2.0. See the LICENSE_MediaPipe file for details.

Documentation

Express:

MediaPipe Documentation:

Input Shapes:

  • FaceDetector: 192 x 192
  • FaceMesh-V2: 256 x 256
  • Blendshape: 1 x 146 x 2
  • HandLandmarker (Full): 192 x 192, 224 x 224
  • Pose detector (Full): 224 x 224 x 3
  • Pose landmarker (Full): 256 x 256 x 3
  • SelfieMulticlass: 256 x 256

About

This project integrates MediaPipe Solutions with Node.js and Express for real-time computer vision tasks. It showcases examples of image segmentation, hand and face detection, and pose detection, with a combined example for all three types of landmark detection.

Topics

Resources

License

Apache-2.0, Apache-2.0 licenses found

Licenses found

Apache-2.0
LICENSE
Apache-2.0
LICENSE_Mediapipe

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published