WebXR AR offscreen capture: 3D zoomed in vs actual AR view (Samsung Android, no camera-access)

🌵 Needle Engine
1

Hi,

Following up on my previous question about AR capture on Samsung Android.

I’m compositing getUserMedia photo + offscreen 3D render.
Projection matrix copy works (objects appear), but 3D is still
zoomed in compared to actual AR view.

Current setup:

  • renderer.domElement: 1080 x 2340 (devicePixelRatio: 2.625)
  • xrCam cameras[0] fovY: 73.9, aspect: 0.462
  • mainCamera.fov: 50, aspect: 0.511
  • takePhoto() resolution: 3060 x 4080
  • WebARSessionRoot scale: [1,1,1], position: [0,0,0]
  • userScale: undefined

What I’m doing:

  1. onAfterRender() - save xrCam pose every frame:
    _lastXRCamPos, _lastXRCamQuat from xrCam.matrixWorld

  2. On capture button:

    • Create offscreen renderer (1080x2340 or 3060x4080)
    • Copy saved pose to snapCam
    • Copy xrCam.cameras[0].projectionMatrix to snapCam
    • Render scene with snapCam
    • Composite with getUserMedia takePhoto()

Result: Camera background matches perfectly, but 3D objects
are zoomed in / appear closer than in actual AR view.

Question: Is there something specific about how Needle Engine
positions the XR camera relative to scene objects that would
cause this mismatch? Is there a recommended way to do offscreen
AR capture in Needle Engine?

Thank you

URL:

2

Hey Sangyeon — the issue is that you’re using xrCam.cameras[0].projectionMatrix but the XR camera in Needle Engine applies an additional offset to the projection matrix for the headset’s IPD and view offset. That offset isn’t baked into cameras[0]'s projection matrix alone.

Try copying the entire matrixWorld of xrCam (position + quaternion) but don’t copy the projection matrix from cameras[0] — instead, use the main camera’s projection matrix with its original FOV and aspect. The XR camera’s cameras[0] projection matrix is already adjusted for the left eye’s viewport, which creates the zoom effect you’re seeing.

To confirm: set your snapCam’s projection matrix to the main camera’s projection matrix (the one that works in the actual AR view), not the XR camera’s. If that fixes the zoom, you’ve confirmed it.

1 replies left - Become PRO for more replies.
Reply to this post directly if you need further help (bottom right button)

3

image
image343×270 42.3 KB

Thank you for the suggestion. I tried using mainCamera.projectionMatrix
instead of xrCam.cameras[0].projectionMatrix, but the 3D is still
zoomed in.

Additional info:

  • mainCamera.fov: 50, aspect: 0.511
  • The downloaded image is 3060x4080 (offscreen renderer size)
    but AR screen is 1080x2340

Two questions:

  1. Should I set offscreen renderer size to 1080x2340 (same as AR screen)?
  2. Is mainCamera.projectionMatrix the same as what’s used
    for actual AR rendering, or does Needle Engine modify it
    when entering XR mode?

Attaching screenshots:
Image 1 = actual AR screen
Image 2 = downloaded capture (still zoomed in)

4

Two issues:

  1. Use 1080×2340 for the offscreen renderer — matching the AR screen resolution avoids the mismatch that causes the zoom. The 3060×4080 capture is higher res but the projection matrix is calibrated for 1080×2340.

  2. mainCamera.projectionMatrix is not what WebXR uses — when entering XR, Needle Engine replaces the camera’s projection with the XR device’s view-specific projection (which includes IPD offset, viewport adjustments, and near/far overrides). You need to capture the actual projection being used per frame. In onAfterRender(), get xrCam.cameras[0].projectionMatrix.clone() — that’s the real per-frame projection. Copy that to your snapshot camera’s projection matrix and set its fov, aspect, near, far to match.

0 replies left - Become PRO for more replies.
Reply to this post directly if you need further help (bottom right button)