Bug: Rendering stereoscopic side-by-side image, Layers only work on Main Camera not WebXRArrayCamera

I’ve tried this dozens of different ways. In my XRCameraController.ts script, layer changes to main camera affects all eyes, even when in VR mode. Any layer changes to WebXRArrayCamera or any WebXRCamera in the array have no effect. I’ve tried various layers to avoid conflicts such as left eye layer 8 and right eye layer 9.

Layer changes on meshes in scene works.

I created a simple three.js page that runs on the same webserver using the same three.js library in the same folder as the needle app. Changes to layers works on its WebXRArrayCamera yielding the desired stereoscopic effect.

I’ve attached everything here.

Zip file of indexvideo.html, images, css that works on vanilla three.js:

XRCameraController.ts (for cameras), SetLayer.ts (for meshes) zipped:

Let me know if you want a copy of my whole Unity project.

Thank you in advance!
Darius

Hi Darius,

Thanks for providing details about the issue with your WebXR setup. It seems like you’re facing issues with layer management for stereoscopic rendering using the WebXRArrayCamera.

To address this, here are some steps you can try:

  1. Ensure Camera and Layer Setup: Verify that your WebXRArrayCamera is correctly set up and that both left and right eye cameras are configured to respect different layers. In your XRCameraController.ts, ensure you’re manipulating the correct cameras corresponding to the right layers.

  2. Layer Assignment: Make sure that you’re not only setting the layer on the camera but also on the meshes in the scene that you want to render for that specific camera. This might include checking the layer settings in the SetLayer.ts script.

  3. Debugging: Log the layers of both the WebXRArrayCamera and your scene objects to ensure they’re being set properly. Sometimes knowing the state of the layers at runtime can give insights into what’s going wrong.

  4. Experiment with Layer Masks: When you set layers for the WebXRArrayCamera, make sure you also adjust the layer masks correctly if your objects are still rendering on the wrong eye. This can often mean setting up masks for the left and right cameras distinctly, depending on their intended visuals.

  5. Different Camera Instances: If both eyes are rendering the same content, check if you are working with separate instances of the WebXRCamera. Each instance should have distinct settings that you configure individually.

If you continue to experience issues, you might consider sharing the exact changes made in your code for deeper inspection or potentially a zip of the whole Unity project if the problem is still elusive.

If you provide your specific XRCameraController.ts and SetLayer.ts files with the relevant portions that handle camera and layers, I can help further!

Feel free to ask if you have additional questions.

1 replies left - Get a PRO license for more AI replies.
thread_xjCV62Cc4n9nh5qDtyD9b9wg
Reply to this message to continue the conversation

// src/scripts/XRCameraController.ts
import { Behaviour, serializable, Context } from “@needle-tools/engine”;
import * as THREE from “three”;

// Define your layers (consider putting these in a shared constants file)
const LEFT_EYE_LAYER = 8;
const RIGHT_EYE_LAYER = 9;
const DEFAULT_LAYER = 0;
// Add any other layers you might use
const UI_LAYER = 5;

export class XRCameraController extends Behaviour {
@serializable()
targetLayer: number = DEFAULT_LAYER;

// Keep track if XR layers were set to avoid redundant logs/operations
private xrLayersSet: boolean = false;

start() {
        this.setCameraLayers();
}

// Optional: Use update or lateUpdate if layers need to change dynamically
// lateUpdate() {
//    // Example: Change layer based on some condition
//    // if (someCondition) {
//    //     this.setCameraLayers(NEW_LAYER);
//    // }
// }

/**
 * Sets the layers for the specified cameras (Main and/or XR).
 * Call this method from other scripts or animations to change layers dynamically.
 * @param newTargetLayer The primary layer the camera should render.
 */
public setCameraLayers() {
    this.xrLayersSet = false; // Reset XR flag if called again

    console.log(`Setting camera layers to target: ${this.targetLayer}, include default: ${this.targetLayer}`);

    // --- Apply to Main Camera ---
    //this.configureCameraLayers(this.context.mainCamera, DEFAULT_LAYER, "Main Camera");
    this.context.mainCamera.layers.disableAll();
    this.context.mainCamera.layers.enable(this.targetLayer);
    this.context.mainCamera.layers.enable(LEFT_EYE_LAYER);
    this.context.mainCamera.layers.enable(RIGHT_EYE_LAYER);

     // --- Apply to XR Cameras ---
    // We often need to do this continuously in update/lateUpdate for XR,
    // as the XR manager might reset layers. The XRCameraController example
    // does this in lateUpdate. If using *this* script *instead* of
    // XRCameraController for layers, you might move the XR part to lateUpdate.
    // For simplicity here, we'll do it once when called.
    this.trySetXRLayers();

}

// Use lateUpdate to potentially override XR settings if needed, similar to XRCameraController
lateUpdate() {
    // If applying to XR, ensure layers are set correctly each frame,
    // especially if another system might be interfering.
    if (this.context.renderer?.xr?.isPresenting && !this.xrLayersSet) {
         this.trySetXRLayers();
    }
     // Reset flag if no longer presenting
    if (!this.context.renderer?.xr?.isPresenting) {
        this.xrLayersSet = false;
    }
}


private trySetXRLayers() {
    const renderer = this.context.renderer;
    if (!renderer) return;
    const xrManager = renderer.xr;

    // Check if XR is active and we have the camera array
    if (xrManager && xrManager.enabled && xrManager.isPresenting) {
        const xrCamera = xrManager.getCamera() as THREE.WebXRArrayCamera;

        if (xrCamera && xrCamera.isArrayCamera && xrCamera.cameras.length >= 2) {
            // Configure Left Eye (typically uses LEFT_EYE_LAYER or the main targetLayer)
            // Decide if XR eyes should see *only* their specific layer + default,
            // or if they should see the general 'targetLayer' + default.
            // Let's assume they should see their specific eye layer + default for stereo separation.
            this.configureCameraLayers(xrCamera.cameras[0], LEFT_EYE_LAYER, "XR Left Eye");

            // Configure Right Eye
            this.configureCameraLayers(xrCamera.cameras[1], RIGHT_EYE_LAYER, "XR Right Eye");

            this.xrLayersSet = true; // Mark as set for this frame/session
            xrCamera.layers.enable(this.targetLayer);
            xrCamera.layers.enable(LEFT_EYE_LAYER);
            xrCamera.layers.enable(RIGHT_EYE_LAYER);
        }
        // Optional: Log if XR is presenting but camera isn't ready yet
        else if (this.context.time.frameCount % 120 === 0) { // Log less frequently
             console.log("CameraLayerController: XR Presenting, but ArrayCamera not valid yet.");
        }
    }
}

/**
 * Helper function to configure layers for a single camera.
 */
private configureCameraLayers(camera: THREE.Camera, layerToEnable: number, cameraName: string) {
    // Start fresh by disabling all layers
    camera.layers.disableAll();

     // Enable the target layer
    camera.layers.enable(layerToEnable);

    // Log the resulting mask for debugging
    console.log(`${cameraName} layers set. Mask: ${camera.layers.mask}`);
}

}

// Updated SetLayer.ts
import { Behaviour, serializable } from “@needle-tools/engine”;
import * as THREE from “three”;

const LEFT_EYE_LAYER = 8;
const RIGHT_EYE_LAYER = 9;
const DEFAULT_LAYER = 0;

export class SetLayer extends Behaviour {
@serializable()
myLayer: number = DEFAULT_LAYER;

start() {
    const obj3d = this.gameObject as THREE.Object3D;

    if (obj3d) {
        // Apply layer recursively to this object and all its children
        obj3d.traverse((child) => {
            // Check if it's a Mesh or potentially other renderable types if needed
            // if (child instanceof THREE.Mesh /* || child instanceof THREE.Line etc. */) {
                child.layers.set(this.myLayer);
            // }
        });

        // Log the mask of the root object this script is attached to for confirmation
        console.log(`${this.gameObject.name} root and children layers set for layer ${this.myLayer}. Root mask: ${obj3d.layers.mask}`);

        // Optional: Log mask of first Mesh child found for deeper debugging
        const meshChild = obj3d.getObjectByProperty("isMesh", true);
        if(meshChild) {
            console.log(`  - First mesh child (${meshChild.name || 'unnamed'}) mask: ${meshChild.layers.mask}`);
        }

    } else {
        console.warn("Could not find Object3D for layer assignment on", this.gameObject.name);
    }
}

}

Hi, that’s default webxr behaviour in threejs.

The webxr camera layers are untouched by Needle Engine and the WebXRManager is a threejs class that handles stereo rendering - three.js synchronizes the camera layers

See the code here in threejs:

Have you tried to check that?

A workaround might be to patch the camera render function and modify render layers in there for the xr camera (or another function to override this behaviour).

Please dont open Multiple posts for the same question: How to render a stereoscopic side-by-side image in a scene for VR viewing? - #7 by socinian

Thanks. I’ll look at that three.js code and make sure it gets called.

I wasn’t sure how to add the “Bug” tag, which is why I created the second post. I’ll be more concise in the future.

Hi, how do you mean be sure it gets called? This code runs every frame - or do you mean you look into how to override it?