How do I apply AR Face tracking/ Filter from Unity to Needle?

🌵 Needle Engine
21

But it does not go full screen with this solution

image.png
image.png2876×1610 317 KB

by user 373346625025736705

22

CSS should take care off fullscreen, when playsInline is false :thinking:

by user 474974683163394049

23

Will it be okay to share your project? :upside_down_face: May be I am missing something else!

by user 373346625025736705

24

sure thing :slightly_smiling_face: I’ll create a github repo during the day and share it here. i’m again at work atm, so it will be very likely evening until you’ll get the link

by user 474974683163394049

25

i also fixed up the local run mode, making it work locally ^^

by user 474974683163394049

26

by user 474974683163394049

27

here you go :slightly_smiling_face: just open the projects/Needle Engine Samples 2020_3 content in Unity and you’re ready to rock!

by user 474974683163394049

28

updated everything and moved the scripts from the “dev” examples into their respective Unity scene, installing the mediapipe npm package and tested locally. that should basically be a “open and run” for you :slightly_smiling_face:

by user 474974683163394049

29

i even adapted the shadowRoot workflow, as this one seemed a little better suited for this kind of things.

by user 474974683163394049

30

Very cool, thanks for sharing! Do you have a deployed link to test this out?

31

Man you are a superhero. I have been following your YouTube channel for a while now. And now this!! You are out of this world :face_holding_back_tears:

by user 373346625025736705

32

Sure thing: https://fdgs-needle-mediapipe.glitch.me/

by user 474974683163394049

33

Thank you :blobaww:

by user 474974683163394049

34

Btw @maXR could you imagine adding some of your stuff to the contributions too? :slightly_smiling_face: Needle Engine Documentation

35

Ui, didn’t know about the contributions Oo I’ll take my time over the next few weeks and add some of my stuff :slightly_smiling_face:

by user 474974683163394049

36

Nice thank you :slightly_smiling_face: would be cool if there’s snippets that are easily shareable / not too much work :slightly_smiling_face:

37

this whole post could end up there as a kind of “recipe” imho

by user 395602247196737546

38

hehe, i guess i should really write a short HowTo into the README.md of the repo. The post itself is unfortunately not very helpful for import and setup of a new project. Hopefully I’ll have time soon, to start writing :sweat_smile:

by user 474974683163394049

39

Here is a small sample script that does what @maXR described here :slightly_smiling_face: it can be put anywhere in the scene

import { Behaviour, ClearFlags, RGBAColor } from "@needle-tools/engine";

export class VideoBackground extends Behaviour {

    async awake() {
        // create video element and put it inside the <needle-engine> component
        const video = document.createElement("video");
        video.style.cssText = `
            position: fixed;
            min-width: 100%;
            min-height: 100%;
            z-index: -1;
        `
        this.context.domElement.shadowRoot!.appendChild(video);

        // get webcam input
        const input = await navigator.mediaDevices.getUserMedia({ video: true })
        if (!input) return;
        video.srcObject = input;
        video.play();

        // make sure the camera background is transparent
        const camera = this.context.mainCameraComponent;
        if (camera) {
            camera.clearFlags = ClearFlags.SolidColor;
            camera.backgroundColor = new RGBAColor(125, 125, 125, 0);
        }
    }
}
40

Here’s a live link: https://linen-upbeat-sailboat.glitch.me/