Ready player me avatars

Is it possible to use Ready Player Me avatars in needle? if so what would be the pipeline to get them set up?

Original Post on Discord

by user 1081953198014734357

or any other human looking avatars in a multiplayer experience?

by user 1081953198014734357

Currently there are no VR avatars ready in the engine or samples.

I would suggest to search for Ready Player Me three.js.
https://medium.com/quark-works/how-to-render-ready-player-me-in-three-js-d96a840ed437
If you skip all the Three.js setup.

It boils down to downgliding a .glb from a remote source, which is quite simple and is showcased here:
https://engine.needle.tools/samples/open=1#external-content

Then you can reparent it / set it during the runtime to your avatar.

There a few IK components mentioned in the docs:
https://engine.needle.tools/docs/xr.html#avatars

Namely here you should be interested into OffsetConstraint which you can setup such that the torso model is driven by the head.

But i see that RPM avatars have neck, so you might probably want to drive the bones.

There’s a planned support / example for VRM Avatar, since it is a robust standard

You can also watch this talk: https://www.youtube.com/live/joQJH-mmEII?feature=share
And that scene is also in our samples repo (in a branch, right @kipash :cactus:?)

ah :bulb: that’s true. I’ll have to refresh my memory on what is on sample branches then :slight_smile: my bad, sorry for misleading.

branch samples/custom-avatars
https://github.com/needle-tools/needle-engine-samples/tree/samples/custom-avatars

Not misleading, you’re completely right that we don’t have direct VRM support (as used by Ready Player Me) so far but it’s planned at some point :slightly_smiling_face:

@kipash :cactus: how do i get this sample into my unity project to try it from this branch?

by user 1081953198014734357

Visit this link https://github.com/needle-tools/needle-engine-samples/tree/samples/custom-avatars

Click the green clone, select download zip.

Then navigate to package/runtime/custom-avatars

Copy the contents to your proj and it should work. :crossed_fingers:

If you’re comfortable with git you can also clone the samples repo and switch to the “samples/custom-avatars” branch, then open the samples project.

so in the sample i’m seeing an avatar being created and you can move it in desktop , but in VR ti doesn’t move along with the head or hands?
didn’t even see hands in the custom hands sample but i might be in the avatar branch

by user 1081953198014734357

Sorry for the silence, i’ll test it out tonight if that is an issue fo the branch :+1: @kipash :cactus:

make sure that the room parameter in the url is the same on Quest and PC, otherwise the users are in different rooms.

The scene contains a player right from start, so if you enter VR it doesn’t seem to hide. So what you are probably seeing is your local player and not the other user.

I’ve just tried and it is working as expected

Potentially, i think a quick fix would be to add XR Flags component to the player model and disable the visibility in VR.

Also confirming that the hands do work on the custom-avatar branch :+1: Do you have hand tracking enabled on the quest?

You can quickly check that if you put down your controllers, and move your hands in front of the hadset if you start seing them or not. If you still see only the controller, you probably don’t have the hand tracking on.

You can find that on Quest 2’s Settings/Movement Tracking/Hand tracking and enable that toggle.

@SoundGuy2

Let me know if you would have any furhter issues, hope that the hand tracking will start working :slight_smile:

I don’t think the sample on the branch currently supports entering VR and having the Avatar move along with your head / hands, which is one of the reasons why it’s still on a branch / not released.