Image Tracking + Extended Tracking

🌵 Needle Engine
1

Hello.
Is it possible to trigger and anchor 3d content to an image tracker, and then switch to extended tracking (ground tracking), with the image tracker out of view?
The use case is for a museum, we’d like to use a QR code to anchor the content to the model, and then allow the user to walk around the museum piece freely with content staying relatively in place.

2

Hello Rupert,

Yes, it is possible to anchor 3D content to an image tracker and then switch to extended tracking mode, allowing users to walk around the museum piece freely with the content remaining relatively in place.

Here’s a step-by-step breakdown of the implementation:

  1. Image Tracking Setup: Use WebXRImageTracking to detect the QR code or image marker.
  2. Object Placement: When the image is detected, anchor your 3D model to the detected image.
  3. Extended Tracking: Once the anchor is established, you can switch to extended tracking, allowing the content to stay in place as the user’s perspective changes.

Sample Code

Here’s an example code snippet demonstrating this approach:

import { WebXRImageTracking } from "@needle-tools/engine";

class MuseumPieceTracker extends Behaviour {
    private imageTracking: WebXRImageTracking;

    awake(): void {
        this.imageTracking = GameObject.findObjectOfType(WebXRImageTracking);
        this.imageTracking.on("image-tracking", this.onImageTrackingUpdate.bind(this));
    }

    private onImageTrackingUpdate(trackedImages: WebXRTrackedImage[]) {
        // Logic to handle anchoring the 3D model
        trackedImages.forEach(image => {
            if (image.state === "tracked") {
                // Anchor object to tracked image, e.g. display model
                const model = image.model; // Your 3D model reference
                if (model) {
                    // Position the model based on tracked image data
                    image.applyToObject(model);
                }
            } else {
                // Handle extended tracking logic
                // Keep the model in its last known position or handle transitions
            }
        });
    }
}

Notes:

  • Ensure that hideWhenTrackingIsLost is set to false (or manage visibility as per your needs) to allow the content to stay at its last known position when the tracker goes out of view.
  • The transition to extended tracking should maintain the last known pose for continued user interaction.

Explore the image tracking sample provided in the Needle Engine documentation for live implementation examples (see /everywhere-actions.md).

If you need further customization or have specific conditions, please let me know!

3 replies left
thread_g3yPYCzLBxxUEHUdUs1kC0vI
Reply to this message to continue the conversation

3

You mention switching to extended tracking but have not given an example, note I am using the Unity SDK

4

Hello Rupert,

In Unity using the Needle Engine SDK, you can implement image tracking along with switching to extended tracking effectively. The key components you will utilize are WebXRImageTracking and the management of object visibility after losing the image.

Implementation Steps:

  1. Setup Image Tracking: Configure your WebXRImageTracking on your AR session to track the QR code or any designated image markers that are part of your museum display.

  2. Anchor the 3D Content: Upon detecting the image, you will want to instantiate or position your 3D content at the location of the marker.

  3. Handling Extended Tracking:

    • Use the hideWhenTrackingIsLost property to manage visibility when the tracking state changes.
    • When the image is no longer tracked, you preserve the last known location of the 3D content so it can remain in place when the user is free to roam.

Sample Code Snippet

Here is a code example on how to switch between image tracking to extended tracking seamlessly:

using Needle.Engine;
using UnityEngine;

public class MuseumTracker : MonoBehaviour
{
    private WebXRImageTracking imageTracking;

    void Awake()
    {
        imageTracking = GetComponent<WebXRImageTracking>();
        imageTracking.onTrackableStateChanged += OnTrackableStateChanged;
    }

    private void OnTrackableStateChanged(WebXRTrackedImage[] trackedImages)
    {
        foreach (var image in trackedImages)
        {
            if (image.state == "tracked")
            {
                // Place your 3D content using image.applyToObject method
                // Example: Instantiate your content and apply to the image
            }
            else if (image.state == "lost")
            {
                // Preserve last-known pose when tracking is lost
                if (image.hideWhenTrackingIsLost == false)
                {
                    // Maintain position of the object
                    // E.g. Set active to true and retain position
                }
            }
        }
    }

    void OnDestroy()
    {
        imageTracking.onTrackableStateChanged -= OnTrackableStateChanged;
    }
}

Notes:

  • hideWhenTrackingIsLost is critical in determining if the object should still be displayed when the image cannot be tracked.
  • This setup ensures that even when the QR code is out of view, the users can still interact with the anchored content, fitting your museum’s use case perfectly.

You may find more details and live implementation examples on the Needle Engine documentation or samples repositories (see the documentation) (see the documentation).

If you have any specific questions on setup or modifications, feel free to ask!

3 replies left
thread_g3yPYCzLBxxUEHUdUs1kC0vI
Reply to this message to continue the conversation

5

Hello @Rupert_Stamp

The WebXR Image Tracking component has an option to keep images visible when they’re not being actively tracked anymore - so it should be enough to just leave this toggle off for the desired effect

image

Please note that at this point in time users on Android devices need to enable the WebXR incubations flag. See Image Tracking | Needle Engine for details