Example of touch start event?

Just as I kept posting here too, I found the root cause was:
this.context.input.convertScreenspaceToRaycastSpace(origin) on iOS WebXR Viewer returns ‘0’ for this.context.domWidth and this.context.domHeight.

Changing those to window.innerWidth and window.innerHeight works correct for WebXR AR with Mozilla WebXR Viewer on iOS

I can confirm the issue happens when using the built in Needle methods:
screenPointToRay(touch.clientX, touch.clientY);
and
this.context.input.convertScreenspaceToRaycastSpace(origin);
https://github.com/needle-tools/needle-engine-support/issues/118#issuecomment-1330915967

by user 103054507105067008

Ah thank you for investigating! It “makes sense” - I didnt think about that but i when debugging last time I noticed that Mozilla XR removed the canvas from the DOM - I thought it only did that falsely on exit AR but that would suggest it’s removing the canvas on enter AR even

Could you try if adding this code bit in engine_setup:135 fixes the issue for you?

    private calculateBoundingClientRect() {
        // workaround for mozilla webXR viewer
        if (this.isInAR) {
            this._domX = window.innerWidth;
            this._domY = window.innerHeight;
            return;
        }
        ...
    }

Will try this first thing tomorrow but I know for sure that workaround works in the local solution I posted in the git issue as I changed the code to do the same with local copies

by user 103054507105067008

I laid it out like this and it produces the NaN error again, reverting back to the example solution in my git issue fixed it again, it seems to be this line in engine_input.ts that needs to be changed:

    convertScreenspaceToRaycastSpace(vec2: Vec2) {
        vec2.x = (vec2.x - this.context.domX) / this.context.domWidth * 2 - 1;
        vec2.y = -((vec2.y - this.context.domY) / this.context.domHeight) * 2 + 1;
    }

to:

    convertScreenspaceToRaycastSpace(vec2: Vec2) {
    vec2.x = (vec2.x - this.context.domX) / window.innerWidth * 2 - 1;
    vec2.y = -((vec2.y - this.context.domY) / window.innerHeight) * 2 + 1;
    }

by user 103054507105067008

ah thanks will change that in the core property then too (domWidth returning innerWidth for AR etc)

ideally we dont have to think about it anymore and they just return the correct values based on the state of the app (innerwidth is not correct in some cases so i can not change that there e.g. when you dont have a fullscreen canvas)

just because of mozilla xr :smile:

Makes sense, I think usually AR is a full screen takeover luckily, might be a non issue if Apple eventually allow WebXR to run in Safari… almost all of the clients use iOS devices unfortunately

by user 103054507105067008

Do you have access to a test iPhone for testing btw? I feel like I’m always hitting weird issues just on iOS WebXR Mozilla when otherwise everything works great on Android Chrome

by user 103054507105067008

We regularly test on WebXR Viewer as well, but it has so many quirks that we don’t always find all of them - super happy that you’re testing this so diligently.

The Internet Explorer of WebXR

Thanks :sunglasses: it really does feel like an abandoned web browser, I am sure there is some arbitrary Apple-only reason why it can exist on its own as a fully working browser that isn’t a reskinned Safari and support WebXR while other browsers don’t

by user 103054507105067008

Oh, it is a reskinned Safari :rofl: they just added some native methods to access ARKit and expose it through relatively hacky JavaScript

That explains how it was allowed to pass App Store review by Apple :joy:

by user 103054507105067008

What’s interesting (and problematic) is that yes, it is abandoned, and they added a ton APIs that were in discussion ~2 years ago - some of which changed, some of which disappeared

If you’re in for an adventure it’s quite interesting to read through their WebXR Shim, copied here for convenience:

Also interesting: if we were to fix things in this script I believe we could actually patch that JavaScript side of things in the WebXR Viewer (they’re literally fetching that from a server at page request)

There’s also the repo of course, may be easier to read in parts, but I’m not sure which exact commit is the “last survivor” currently live:

For example, theoretically it can do Image Anchors, which isn’t standardized yet at all