Custom VR and AR Controls

🌵 Needle Engine
1

Hi, I am trying to use my own custom Controls to move in an VR world und want use the same controls in immersive AR and different ones on mobile AR. I am using the webXR component to get the Controller and their position(, rotation, scale, and buttons webXR?.RightController?.input?.gamepad?.buttons[1].pressed). Works fine in VR. but when I try in AR (on my quest 2 with passthrough) the hole webXR component is undefined and I cant get any input from my Controllers (I could get the position from: this.context.renderer.xr.getController(0).getWorldPosition(new Vector3()); but it only works when there was already a VR Session and I am not getting the buttons data)
I also tried getting the webAR component but that doesn’t seem to work either.

let webXR = GameObject.getComponent(this.webXR3D, WebXR);
//let webAR = GameObject.getComponent(this.webXR3D, WebAR);
let currentPositionLeft;// = webXR?.LeftController?.controller.getWorldPosition(new Vector3());
currentPositionLeft = this.context.renderer.xr.getController(0).getWorldPosition(new Vector3());

let currentPositionRight;// = webXR?.RightController?.controller.getWorldPosition(new Vector3());
currentPositionRight = this.context.renderer.xr.getController(1).getWorldPosition(new Vector3());

TLDR;
I would like a way how I can reliable use the right and left controller whenever they are available and would like to know when i am in mobile AR, immersive AR, VR or normal monitor setup (webXR.IsInAR seems to be undefined)

thank you for your help

Original Post on Discord

by user 253912725917401089

2

If WebXR is undefined then that’s most likely because you dont have a correct reference to the component.

this.context.xrSession gives you the session if it exists, there’s also this.context.isInAR and this.context.isInVR and this.context.isInXR that you can use.

If you want to get access to the XRFrame you can implement onBeforeRender(frame:XRFrame) in your component and use it directly

3

WebXR.Controllers gives you access to a list of all the XR controllers that are created for AR and VR. LeftController and RightController do only return one for VR right now I think as you said because the controllers in AR don’t have handedness (maybe execpt for pass-through VR - not sure about that)

4

Yes, thank you this.context.xrSession?.inputSources[0]?.gamepad?.buttons works much better than webXR?.LeftController?.input?.gamepad?.buttons and this.context.isInAR better than webXR?.IsInAR thats all i want. Thank you for your generally fast responses in this forum!

by user 253912725917401089

5

Interesting - I would assume that webXR is undefined there? :thinking: Is that possible? Can you check? Because I dont see how it can be undefined in code

6

image.png
image.png1689×224 47.4 KB

by user 253912725917401089

7

In immersive AR or Quest 2 Oculus browser on passthrough

by user 253912725917401089

8

What does it output when you just console.log(webXR) ? It will print undefined too if webXR is undefined (because of the ?) in which case it would have nothing to do with the AR state.

9

image.png
image.png
image.png

by user 253912725917401089

10

ì get the WebXR object with this line ```@serializable(Object3D)
public webXR3D : Object3D| null = null;

let webXR = GameObject.getComponent(this.webXR3D, WebXR);```and then dragging the main XR gameobject in unity from the sample scene to the reference

by user 253912725917401089