Hand tracking

🌵 Needle Engine
1

Can I use hand tracking without VR Headset to create try on for wrists?

2

Hi Saad Baig! To determine if you can use hand tracking without a VR headset in Needle Engine for creating a wrist try-on experience, a few considerations need to be taken into account:

  1. Device Compatibility: Check if your device supports hand tracking. Some devices, like certain smartphones or tablets, have built-in hand tracking capabilities.

  2. Web Frameworks/Tools: Utilize existing web frameworks or tools that enable hand tracking in a browser environment without needing a VR headset. Libraries such as Mediapipe can provide hand tracking features for web applications.

  3. Integration with Needle Engine: Implementing hand tracking might require integrating specific APIs or libraries that support this functionality into your Needle Engine project.

To proceed, an overview of how you would typically integrate these components is below:

  1. Use Mediapipe for Hand Tracking:

    • Set up Mediapipe in your project to track hands using a webcam.
    • Use the hand landmark data to create a 3D model overlay for try-ons.

  2. Integrate with Needle Engine:

    • Import the hand tracking data into Needle Engine.
    • Use @needle-tools/engine to control 3D models based on the tracked hand positions.

  3. Rendering the Wrist Models:

    • Access and manipulate 3D models of wrist accessories within Needle Engine.
    • Update model positions in real-time based on the hand tracking data.

If you have specific files or code references that you’re working with, let me know so I can assist further! Additionally, ensuring Needle Engine and any third-party libraries you use are compatible with your current setup is crucial. You can refer to the Needle Engine documentation for guidance on setting up custom components. Would you like any further details or specific help on any of these steps?

24 replies left
thread_BaBHAnBSfJdkoBvNRkCpZ5D6
Reply to this message to continue the conversation

3

Hello, this is related to Plane tracking & body tracking XR mode via phone camera? - #6 by marcel - by using mediapipe this could be added. Either by you if your familiar with coding or by adding it to Needle Engine core. Please vote for the roadmap items you’d like to see or need so we prioritize :slight_smile:

1 Like