Open AR Scene Directly without XR mode

🌵 Needle Engine
1

Hello,

We’re looking to reduce the number of UI buttons in our project and simplify the user experience. Specifically, we’d like to start an AR session by clicking on a 3D “hero” object within the scene—rather than using a separate button.

We currently have one scene with 3 hero objects displayed in XR mode. Our goal is to open a new scene in AR mode on mobile phones (both iOS and Android) when one of the heroes is clicked. How can we achieve this? Is there an existing example for this kind of flow, or do we need to build it from scratch?

The main focus is to automatically enter AR mode when the new scene loads, without requiring any additional clicks or UI interactions.

We’d like to keep everything as simple as possible, so any tips on reducing scene transitions or minimizing UI interactions would be greatly appreciated.

2

Hello Nemanja,

To achieve your goal of starting an AR session by clicking on a 3D hero object without needing a separate button, you can follow these steps:

  1. Detect Click on the Hero Object: Use a raycaster to detect clicks on your 3D models in the scene.
  2. Transition to New Scene in AR Mode: Once a hero object is clicked, load the next scene and initiate AR mode immediately.

Here’s an example outline in pseudocode for your implementation:

1. Initialize raycaster for detecting clicks.
2. On mouse click:
   a. Check which hero object was clicked using raycaster.
   b. If a hero object is clicked:
      i. Load the AR scene.
      ii. Use XR API to request AR session.
3. In the AR scene:
   a. Automatically enter AR mode upon loading.

Now, here’s a sample code snippet demonstrating how you might implement this in Needle Engine. Replace HeroObject with your specific object names:

import { XRSession, SceneManager } from "@needle-tools/engine";
import * as THREE from 'three';

// Initialize raycaster and variables
const raycaster = new THREE.Raycaster();
const mouse = new THREE.Vector2();

// On mouse click event
function onMouseClick(event) {
    // Calculate mouse position in normalized device coordinates
    mouse.x = (event.clientX / window.innerWidth) * 2 - 1;
    mouse.y = -(event.clientY / window.innerHeight) * 2 + 1;

    // Use raycaster to find intersected objects
    raycaster.setFromCamera(mouse, camera);
    const intersects = raycaster.intersectObjects(heroObjects); // heroObjects is an array of your 3D hero objects

    if (intersects.length > 0) {
        // Load your AR scene here
        SceneManager.loadScene("ARScene"); // assuming ARScene is your new scene
        enterARMode();
    }
}

// Function to enter AR mode
function enterARMode() {
    XRSession.request({
        requiredFeatures: ['local-floor', 'hit-test'],
    }).then((session) => {
        // Start AR session
        // Additional session handling code can go here
    }).catch((error) => {
        console.log("Error entering AR mode: ", error);
    });
}

// Add event listener for mouse clicks
window.addEventListener('click', onMouseClick, false);

Additional Tips:

  • Make sure your AR scene is configured correctly to support AR mode without user interactions.
  • Consider using optimized 3D models for the hero objects to ensure smooth performance.
  • Test thoroughly on both iOS and Android devices to ensure compatibility.

For more specific examples and details, you can refer to the Needle Engine samples repository.

Let me know if you need any more help or clarification!

1 replies left - Get a PRO license for more AI replies.
thread_OGuoU739rrvocK5LvT5QlRr1
Reply to this message to continue the conversation

3

Hi, you can do this with a few lines of code or a custom component.

import {
  Behaviour,
  PointerEventData,
  NeedleXRSession,
} from '@needle-tools/engine';

export class EnterAR extends Behaviour {
  onPointerClick(_args: PointerEventData) {
    console.log('CLICK');
    NeedleXRSession.start('ar');
  }
}

Regarding:

“The main focus is to automatically enter AR mode when the new scene loads, without requiring any additional clicks or UI interactions.”

Keep in mind that the browser enforces(!) that you interact with the page by a click for example before immersive sessions are allowed to start. This is nothing we can change and will be the same in any web application. E.g. Audio is also not allowed to play when you enter the page immediately as a security measure.

Here’s another example (A bit older) where AR is started when you click on the page anywhere

Note that nowadays you need less code than this example for AR NeedleXRSession.start("ar") will work for both Quicklook and WebXR

1 Like
4

This topic was automatically closed 13 days after the last reply. New replies are no longer allowed.