Multi Camera Application

Scene with four active cameras. A main camera and three camera streams in one Scene. Is that possible?

Original Post on Discord

by user 684154017479524352

Hey :wave: what do you mean by “camera streams”? Do you mean streams from users that are streaming their screens or do you mean local cameras in your scene?

local cameras in my scene

by user 684154017479524352

You can refer to the code present in this sample:

Scripting Snippets

https://engine.needle.tools/samples/?open=1#scripting-snippets

Ideally i would advice for usage of render textures, but it seems they have incorrect / unexpected colorspace → resulting in a darker imagine majority of the time. I’ll make a task to investigate :cactus:

Let me know if you would get stuck on anything or if anything wouldn’t be clear :slight_smile:

nice, THX :slightly_smiling_face:

by user 684154017479524352

This is exactly what i need. And it´s works fine. What do you think? Do the different streams affect performance? And if so, how much is the impact?

by user 684154017479524352

Extra camera that renders the scene will definitly impact performance and we are in the land of WebGL and often performance weak devices. But it also depends on your overall usecase and target audience/target hardware.

If you expect people to use it on their iphones, you should do the performance tests there and validate that whatever you want to do is possible. If you would pull up some older hardware, like Samsung S7, you might see that the performance there is bad. (Just an example)

Then based on your use case, you can think about how to simplify the performance impact. Do all 3 cameras need to be rendered every frame? → You can potentially disable them when they are not needed and using the last rendered frame. And so on.

It really comes down to how much you plan to spent on the rest of the scene / features.

in my case it is necessary for all cameras to show their view simultaneously. I have to test it and then see how it really feels. Thank you and best regards Matthias

by user 684154017479524352

You can report your findings here :slight_smile: It is useful for us and others to see what is and isn’t possible :cactus: best of luck :slight_smile:

Regarding performance, depending on what’s shown in these views there may not be a performance impact. E.g. if all the scenes are independent then the resulting performance impact should be comparable to a single camera rendering all the objects together.

Some additional notes: the sample linked above uses render textures, which do have an additional performance impact; a faster way for your particular case (screen-aligned rectangles as output) could be setting the viewport/scissor and rendering those separate cameras manually. See https://threejs.org/docs/#api/en/renderers/WebGLRenderer.setViewport. When you don’t run into performance issues, want to have rounded corners or display the images spatially, render textures should be fine :slightly_smiling_face:

I’m in the process of testing. If nothing happens in the scene, navigation is smooth. If I play a simple animation, I notice a reduction in the frame rate. This increases the more Render to Texture objects are visible.

Splitscreen sounds good. Let’s see if I can solve the issue. Thank you

by user 684154017479524352

Unfortunately, I’m not a programmer, but I’m in the process of learning. That’s why I ask you what you would do to make the script work properly.

import { Behaviour, serializable } from “@needle-tools/engine”;

@serializable
export class SplitScreen extends Behaviour {
private cameras: Camera = ;

// List of cameras to be used in the scene
public cameraList: Camera = ;

start() {
// Load the cameras from the list
this.cameras = this.cameraList;
// Check if at least one camera is present
if (this.cameras.length > 0) {
// Split the screen into four parts
this.setSplitScreen();
}
}

private setSplitScreen() {
// Calculate the size of each camera viewport
const width = Screen.width / 2;
const height = Screen.height / 2;

// Set the position and size of each camera viewport
for (let i = 0; i < this.cameras.length; i++) {
  const camera = this.cameras[i];
  const x = i % 2 === 0 ? 0 : width;
  const y = i < 2 ? height : 0;
  camera.rect = new Rect(x, y, width, height);
}

}
}

by user 684154017479524352

Are the same objects visible in multiple views? Then yes, performance will +1 for each additional render texture

What do you think, is that the right approach? Cameras can be assigned, but no split screen is displayed yet. I’m grateful for every tip. As I said, I am a beginner in programming. THX

import { Behaviour } from “@needle-tools/engine”;
import * as THREE from “three”;

export class SplitScreen extends Behaviour {
private renderer: THREE.WebGLRenderer;
private cameras: THREE.Camera = ;

// List of cameras to be used in the scene
public cameraList: THREE.Camera = ;

constructor(renderer: THREE.WebGLRenderer) {
super();
this.renderer = renderer;
}

start() {
// Add cameras to the cameraList property programmatically
const camera1 = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
const camera2 = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
this.cameraList.push(camera1, camera2);

// Load the cameras from the list
this.cameras = this.cameraList;
// Check if at least one camera is present
if (this.cameras.length > 0) {
  // Split the screen into four parts
  this.setSplitScreen();
}

}

private setSplitScreen() {
// Calculate the size of each camera viewport
const width = this.renderer.domElement.width / 2;
const height = this.renderer.domElement.height / 2;

// Set the position and size of each camera viewport
for (let i = 0; i < this.cameras.length; i++) {
  const camera = this.cameras[i];
  const x = i % 2 === 0 ? 0 : width;
  const y = i < 2 ? height : 0;
  this.renderer.setViewport(x, y, width, height);
  this.renderer.render(camera, camera.userData.target);
}

}
}

by user 684154017479524352

I’ll take a look
By the way, tip for posting here: if you wrap code in three backticks (`) you can make nicely readable code blocks:

import { Behaviour } from "@needle-tools/engine";
import * as THREE from "three";

export class SplitScreen extends Behaviour {
  private renderer: THREE.WebGLRenderer;
  ...
}

image.png