Example of touch start event?

šŸŒµ Needle Engine
21

Just as I kept posting here too, I found the root cause was:
this.context.input.convertScreenspaceToRaycastSpace(origin) on iOS WebXR Viewer returns ā€˜0ā€™ for this.context.domWidth and this.context.domHeight.

Changing those to window.innerWidth and window.innerHeight works correct for WebXR AR with Mozilla WebXR Viewer on iOS

I can confirm the issue happens when using the built in Needle methods:
screenPointToRay(touch.clientX, touch.clientY);
and
this.context.input.convertScreenspaceToRaycastSpace(origin);
https://github.com/needle-tools/needle-engine-support/issues/118#issuecomment-1330915967

by user 103054507105067008

22

Ah thank you for investigating! It ā€œmakes senseā€ - I didnt think about that but i when debugging last time I noticed that Mozilla XR removed the canvas from the DOM - I thought it only did that falsely on exit AR but that would suggest itā€™s removing the canvas on enter AR even

23

Could you try if adding this code bit in engine_setup:135 fixes the issue for you?

    private calculateBoundingClientRect() {
        // workaround for mozilla webXR viewer
        if (this.isInAR) {
            this._domX = window.innerWidth;
            this._domY = window.innerHeight;
            return;
        }
        ...
    }
24

Will try this first thing tomorrow but I know for sure that workaround works in the local solution I posted in the git issue as I changed the code to do the same with local copies

by user 103054507105067008

25

I laid it out like this and it produces the NaN error again, reverting back to the example solution in my git issue fixed it again, it seems to be this line in engine_input.ts that needs to be changed:

    convertScreenspaceToRaycastSpace(vec2: Vec2) {
        vec2.x = (vec2.x - this.context.domX) / this.context.domWidth * 2 - 1;
        vec2.y = -((vec2.y - this.context.domY) / this.context.domHeight) * 2 + 1;
    }

to:

    convertScreenspaceToRaycastSpace(vec2: Vec2) {
    vec2.x = (vec2.x - this.context.domX) / window.innerWidth * 2 - 1;
    vec2.y = -((vec2.y - this.context.domY) / window.innerHeight) * 2 + 1;
    }

by user 103054507105067008

26

ah thanks will change that in the core property then too (domWidth returning innerWidth for AR etc)

27

ideally we dont have to think about it anymore and they just return the correct values based on the state of the app (innerwidth is not correct in some cases so i can not change that there e.g. when you dont have a fullscreen canvas)

28

just because of mozilla xr :smile:

29

Makes sense, I think usually AR is a full screen takeover luckily, might be a non issue if Apple eventually allow WebXR to run in Safariā€¦ almost all of the clients use iOS devices unfortunately

by user 103054507105067008

30

Do you have access to a test iPhone for testing btw? I feel like Iā€™m always hitting weird issues just on iOS WebXR Mozilla when otherwise everything works great on Android Chrome

by user 103054507105067008

31

We regularly test on WebXR Viewer as well, but it has so many quirks that we donā€™t always find all of them - super happy that youā€™re testing this so diligently.

32

The Internet Explorer of WebXR

33

Thanks :sunglasses: it really does feel like an abandoned web browser, I am sure there is some arbitrary Apple-only reason why it can exist on its own as a fully working browser that isnā€™t a reskinned Safari and support WebXR while other browsers donā€™t

by user 103054507105067008

34

Oh, it is a reskinned Safari :rofl: they just added some native methods to access ARKit and expose it through relatively hacky JavaScript

35

That explains how it was allowed to pass App Store review by Apple :joy:

by user 103054507105067008

36

Whatā€™s interesting (and problematic) is that yes, it is abandoned, and they added a ton APIs that were in discussion ~2 years ago - some of which changed, some of which disappeared

37

If youā€™re in for an adventure itā€™s quite interesting to read through their WebXR Shim, copied here for convenience:

38

Also interesting: if we were to fix things in this script I believe we could actually patch that JavaScript side of things in the WebXR Viewer (theyā€™re literally fetching that from a server at page request)

39

Thereā€™s also the repo of course, may be easier to read in parts, but Iā€™m not sure which exact commit is the ā€œlast survivorā€ currently live:

40

For example, theoretically it can do Image Anchors, which isnā€™t standardized yet at all