Example of navigation + interaction using VR?

Hi guys, I need to setup a webgl app with the following requirements:

  • Navigation of a 3D space using VR (e.g. Oculus Quest) with fallback to normal 3D (keyboard/mouse or touch on mobile).
  • Interaction with environment using VR controllers (e.g. Oculus Quest).
  • Multiplayer

Can you suggest an example or documentation as a good starting point for this? :rocket:

Original Post on Discord

by user 832577308644212766

Hello :wave:

Needles comes built-in with both multiplayer and VR support.

I would suggest opening up the Sandbox sample: https://engine.needle.tools/samples/collaborative-sandbox/?room=needle217
That sample showcases: interactions, VR/AR support and multiplayer capability.

You are able to download the samples and then open the Sandbox sample. Follow the instructions here: https://github.com/needle-tools/needle-engine-samples

In order to connect to the same room, please open the same url on both devices. The URL contains the room parameter which is how the two devices will end up in the same sessions.

For more information on:
VR - https://engine.needle.tools/docs/xr.html
Multiplayer - https://engine.needle.tools/docs/networking.html

Let me know if you have further questions :slight_smile: :cactus:

Thanks @kipash :cactus: ! I have loaded the collaborative-sandbox sample as you suggested. As first learning task I would like to modify the Oculus controller behaviour when turning around. Right now it works at “steps”, I would like to make it continuous. I see there are an XR object with an WebXR.ts script attached and a XRRig object with XRRig.ts script attached. Should I modify one of these scripts? Any guidance? :pray:

by user 832577308644212766

You need write your component that reads the input and applies the rotation :slight_smile:

For reference on coding, i would advice reading the whole Scripting section of the docs (https://engine.needle.tools/docs/getting-started/typescript-essentials.html) - please see the side panel and read the other “articles”.

Fundamentally, you need to reference the WebXR component and there get the controlls from there. On the controller you need to disable the default controls by setting the enableDefaultControls to false.
Then in the input object, you can find the gamepad. From that you can fetch the Joystick axes values.

There’s also this community contribution that fetched input from the controller: https://engine.needle.tools/docs/community/contributions/web3kev/vertical-move-in-vr-using-the-right-joystick-quest/

We are working on improved docs and samples for this, but that needs a bit more work :slight_smile:

Thank you for the great info! I see that I have to create a typescript file exporting a class deriving from Behaviour right? Also the file name must match the class name, otherwise I get an error. Once the typescript file is saved, there is a c# file automatically created in Assets/Needle/Components.codegen. Can you elaborate a bit why this file is needed?

by user 832577308644212766

Yes, your component needs to derive from Behaviour which is an equivalent to MonoBehaviour in unity.

In Needle the name of the typescript class and the .ts file do not need to match. But i would advice to do so for project organization :slight_smile:

C# files are generated from your typescript files so you can add the component on to a Gameobject.

These C# auto generated files only contain public variables and functions so you can set the variables in inspector or call functions from unity events (like on UI button).
Then that data is set to the typescript instance allowing a seamless workflow between unity and the web.

The auto generated files do not contain any contents of the functions, in other words, they don’t contain any logic. Only a data oriented stub is generated to allow interconnectivity.

These auto generated C# classes are partial, so you can always extend the class. That is helpful if you for example want to use needle only for the web and still use unity for native platforms.
This way you can write the component both in C# and Typescript and then export to any platform as you need.

Just to make sure, no C# class is converted into Typescript and No typescript class is converted into C#, only the variable and function declaration are extracted, not their logic.

This is also worth reading → Needle Engine Scripting | Needle Engine Documentation

Thanks a lot guys! I was trying to access the WebXR object in the start() hook to get the controllers, but it seems the array is empty. I guess is because the ENTER VR button is not clicked yet? In this case, I would need an hook for when entering the VR session to get the controllers, but looking here Needle Engine Scripting | Needle Engine Documentation I don’t see a lifecycle method for this. How would you procede?

by user 832577308644212766

WebXR.addEventListener(WebXREvent.XRStarted, () => {});

You’re right, until you enter XR, you won’t be able to interact / configure the controllers.

You can hook to the static events the WebXR component provides like so ^^^.

Ooh I see! Do you know if these events are supported in the WebXR emulator? https://chrome.google.com/webstore/detail/webxr-api-emulator/mjddjgeghkdijejnciaefnkjmkafnnje?ref=blog.mozvr.com

by user 832577308644212766

Yes, they are :+1:

I would recommend the Meta’s emulator insteadhttps://developer.oculus.com/blog/webxr-development-immersive-web-emulator/ for VR development :slight_smile:

Because I have tried the following code but the WEBXR session started log is never triggered when entering the VR session:

export class CustomScript extends Behaviour {
    public webXR: WebXR | null = null;

    start(): void {
        console.log('S T A R T');        
        this.webXR = GameObject.findObjectOfType(WebXR);                
        
        if (this.webXR) {
            console.log(`Adding listener to WebXR object: ${WebXREvent.XRStarted}`);

            this.webXR.addEventListener(WebXREvent.XRStarted, () => {
                console.log('WEBXR session started!', this.webXR);
                this.webXR!.Controllers.map(e => {                    
                    e.enableDefaultControls = false;
                })
            });
        }
    }
}

by user 832577308644212766

Thanks for the suggestion, going to install it asap

by user 832577308644212766

Can you please try to add the listener via the static way?
WebXR.addEventListener(WebXREvent.XRStarted, () => {

It can be an oversight from out end.

Sure, I am trying right now

by user 832577308644212766

Yeah with the static way works!

by user 832577308644212766