VRM, IK, Blend Shapes or Spring Bone support

Hello, I’m looking to use Needle Tools with a project I’m working on that involves use of VRM models, or at least Skinned Mesh Renderers with some limb IK, spring bones, and blendshapes. The VRM standard is pretty widespread and covers all of these though so hoping to use that if possible. I’ve read here that it’s on the roadmap, but I’m wondering if there are any updates on that front or if there’s an alpha release with these features in progress I could try out!

Original Post on Discord

by user 355135788482166784

Hello there – we have an experimental AI bot :robot: that might be able to help you with your question. Would you like to try this out?

How it works:
Reply to the messages of the bot that end with a thread_id to interact with it and ask follow-up questions. You can also mention the bot directly in this channel to get help.

Help requested: please wait a moment…

Here is what I found for you :brain: Discord

Hello kyledapp, the current documentation doesn’t mention explicit support for VRM models; however, Needle Engine provides basic support for avatar functionalities such as blinking and simple limb IK. The AvatarBlink_Simple component randomly hides GameObjects that represent eyes to mimic blinking, and OffsetConstraint can shift objects in relation to another, for example, to level rotation between head and body. Moreover, BasicIKConstraint offers minimalistic IK for simple arm or leg chains (see the documentation).

For more detailed IK, blend shapes, and spring bones, there may not be direct equivalents in the existing documentation or alpha releases. However, Needle Engine does support Skinned Mesh Renderers, which may allow for the creation of similar functionality with custom components. More advanced features could potentially be on the roadmap, and you might want to reach out directly to the Needle Tools team for up-to-date information on VRM support or if an alpha release with these features exists.

1 messages left GPT 4
Thanks for your feedback!

Hey @kyledapp , there is a runtime VRM loading sample branch in our samples repo on GitHub — that could be a good starting point while we implement the rest of the flow

It’ll load the VRM, blendshapes and all data, but on the branch it doesn’t build spring bones or IK automagically right now, you’d have to add that currently

Indeed it’s on the roadmap to support VRM more natively - we just landed our big WebXR refactor that will make it easier for both us and you to support features like this

thanks so much @herbst🌵 , i’ll check that out!

by user 355135788482166784

when you say “doesn’t build spring bones or IK”, do you mean needle has spring bone and IK components and I’d have to just instantiate and hook the data from the VRM up myself, or that I’d have to write my own spring bone and IK components?

by user 355135788482166784

hmm, may have been an issue with how i cloned the samples, but i am seeing this error:

Uncaught SyntaxError: The requested module '/node_modules/.vite/deps/@needle-tools_engine.js?v=39a09144' does not provide an export named 'addCustomExtension' (at VRMLoader.ts:41:50)

by user 355135788482166784

ah i see, i think the example is a little out of sync with the current state of support, addCustomExtension is now addCustomExtensionPlugin and the data structure of the loaded scene is a bit different. but it does load with some tweaks!

by user 355135788482166784

Ah, glad you figured it out. Yes, there was an API change recently and the branch hadn’t been updated

Feel free to make a PR!

Right now you’d have to make them yourself or use an existing three.js lib that supports loading it/has them