Low performance when using post processing in comparison to vanilla threejs

[krizzl0r]

Hi, I’m currently doing some benchmarking to get a feeling for what’s possible using Needle. Interestingly, performance seems to be impacted by postpro effects much more than a vanilla threejs scene is.

It does not really matter what effect is active, as long as the composer is active, frames go down.

Here’s the “minimal” scene in 4k, I just added three cubes, disabled the directional light and added a Volume with just tone mapping active:

Frames drop from solid 60 to 40-45.

Also, the CPU and GPU load goes up and there’s an extra System process using GPU for whatever reason:

I compared it to three.js examples which i modified on the fly to be displayed in fullscreen mode as well and confirmed both apps are using the same drawingBufferSize of 4k.

Here’s the three.js scene using bloom and tone mapping

Solid 60fps.

Also the CPU and GPU load is much more relaxed. Also, the System process isn’t consuming any GPU resources.

I’m running NE 4.2.6 (as 4.3.0 has the black screen bug when using the Volume component), exporting from Unity 2022.3.x. GPU is a 4070s.

[needle-assistant (Needle Engine Assistant)]

Hi Krizzl0r,

Thank you for providing detailed information about your benchmarking results. It seems that you’re experiencing performance drops when using post-processing effects with Needle Engine compared to a vanilla Three.js setup. Here are a few aspects to consider for your investigation:

1. Post-processing Workflow: Review how the Needle Engine integrates the post-processing pipeline. Differences in the setup of the composer or the way effects are applied can lead to performance impacts.

2. Asset Management: Ensure that your assets, including textures used in your post-processing, are optimized for performance. Large textures or complex materials can drastically affect rendering speed, especially at 4K resolutions.

3. Volume Component Settings: While you’ve indicated that only tone mapping is active, double-check for any additional settings in the Volume component that might be contributing to the load.

4. Profiler Tools: Use GPU and CPU profiler tools to trace where the bottleneck is occurring. Tools like Chrome’s DevTools or NVIDIA Nsight can give insights into frames per second and rendering times.

5. Post-processing Effects: Test with a single post-processing effect activated at a time, if not done already. This helps identify if there’s a particular effect that significantly impacts the performance.

6. Update Needle Engine: If the performance issue persists in version 4.2.6 and you are aware of the black screen bug in 4.3.0, consider checking if any commits/fixes have been made that might address performance or compatibility.

7. Unity Export Settings: Make sure that the export settings from Unity are optimized for performance when using Needle.

If you’ve already tested all of the above, you might consider sharing your scene setup or specific configurations that are being used in Needle with post-processing to further identify potential optimization areas.

You can also explore the documentation and sample repositories for performance tips: Needle Engine API and Samples Repository.

Let me know if you need further assistance or specific code adjustments!

1 replies left - Get a PRO license for more AI replies.
thread_hvwLQDSpjleOySZIuuvedgGk
Reply to this message to continue the conversation

[krizzl0r]

For completeness sake, here’s the same scene with the Volume disabled:

60fps, and lower resource consumption of course:

[marwie1 (marwi)]

Could you compare with pmndrs postprocessing? That’s the library we’re using: GitHub - pmndrs/postprocessing: A post processing library for three.js.

[krizzl0r]

Of course, my bad. I thought the examples where referencing the same postpro package.

[krizzl0r]

One thing I noticed is you’re instancing the WebGLRenderer with an antialiasing option. For better comparability I tried to disable it by invoking the following using the console:

Needle.Context.DefaultWebGLRendererParameters.antialias = false;
Needle.Context.Current.createNewRenderer();
Needle.Context.Current.restartRenderLoop();

(Without really knowing what I’m doing of course)

[marwie1 (marwi)]

Hi @krizzl0r thanks for highligthing the issue

The performance seems to drop significantly with multisampling enabled in the EffectComposer:

Can you try if setting multisampling to 8 in your test will cause the same performance drop?

[krizzl0r]

Sure, what’d be the best way to quickly modify it?

[marwie1 (marwi)]

Ah sorry i meant in the postprocessing package examples but maybe that’s not exposed.

In Needle Engine you can set this.context.composer.multisampling = 0

[krizzl0r]

Yep, that’s it.

I had been looking for a way to disable it but I couldn’t find anything anywhere (docs/code/forum/discord). Guess because I was searching for antialiasing and not multisampling…

My findings while playing with it:
Any kind of multisampling instantly kills performance. I know this is kind of expected on high fill rates but this is pretty severe. Even just setting it to 1 (which does not visually help in any way) drops the framerate below 50 and I can hear my fans spin.

But what’s even more interesting: When there’s no composer active (no Volume), the rendered scene has pretty good AA and good performance.
My hack from the post above using Needle.Context.DefaultWebGLRendererParameters.antialias = false; and restarting the renderer does not disable the AA in this case either.
From the looks of it it might be some FXAA pass injected somewhere if no composer is active?

[marwie1 (marwi)]

Currently looking into it as well - when a composer is active the composer does render the scene. Looking at the pmndrs postprocessing demos seems like they always have not set multisampling at all and instead use SMAA (which is added by the Antialias component in Needle Engine). But the demos also haven’t been updated in 2 years so currently looking at the source code.

[krizzl0r]

Toying around with SMAA as well. The current implementation of the AntialiasingEffect component lacks configurability though. It doesn’t even apply the currently single exposed parameter (preset) to the underlying effect

[marwie1 (marwi)]

When you update preset.value to something else it does update the preset. But yes not more is exposed than the presets

[krizzl0r]

Sorry, yes, preset works.

In the meantime I’ve also exposed the edge detection mode and found DEPTH performs visually worst in my use case. COLOR gives a much better result. It’s also the underlying SMAAEffect’s default and should be used here too I reckon.

[krizzl0r]

Back to the original issue: I guess we still need some way to control AA when using the composer. As well as when not using the composer

[marwie1 (marwi)]

Without composer your reply above was correct:

Needle.Context.DefaultWebGLRendererParameters.antialias = false;
Needle.Context.Current.createNewRenderer();

And instead of calling createNewRenderer you can just do the above in global scope (not from within a class or component but right at the top of main.ts for example).

Regarding AA: We’ll probably expose a setting for multisample steps in the Volume component but still looking into it

[krizzl0r]

Based on the AntialiasingEffect component for now this helps with avoiding the MSAA performance trap and setting up SMAA:

namespace Needle.Engine.Components.PostProcessing
{
    public partial class SMAAEffect : PostProcessingEffect
    {
        public enum QualityLevel
        {
            LOW = 0,
            MEDIUM = 1,
            HIGH = 2,
            ULTRA = 3
        }


        public enum Mode
        {
            COLOR = 0,
            DEPTH = 1,
            LUMA = 2
        }

        public enum PredicationMode
        {
            DISABLED = 0,
            DEPTH = 1
            // CUSTOM,
        }

        public Mode edgeDetectionMode = Mode.DEPTH;
        public QualityLevel qualityLevel = QualityLevel.LOW;
        public PredicationMode predicationMode = PredicationMode.DISABLED;
        public bool surpressMsaa = true;
    }
}

import { EdgeDetectionMode as _EdgeDetectionMode, PredicationMode as _PredicationMode, SMAAPreset } from "postprocessing";
import { MODULES } from "@needle-tools/engine/lib/engine/engine_modules";
import { EffectProviderResult, PostProcessingEffect, registerCustomEffectType, serializable, VolumeParameter } from "@needle-tools/engine";

export enum QualityLevel {
    LOW = 0,
    MEDIUM = 1,
    HIGH = 2,
    ULTRA = 3
}

export enum EdgeDetectionMode {
    COLOR = 0,
    DEPTH = 1,
    LUMA = 2
}

export enum PredicationMode {
    DISABLED = 0,
    DEPTH = 1
    // CUSTOM,
}

/**
 * @category Effects
 * @group Components
 */
// @dont-generate-component
export class SMAAEffect extends PostProcessingEffect {
    get typeName(): string {
        return "SMAAEffect";
    }

    @serializable(VolumeParameter)
    readonly edgeDetectionMode: VolumeParameter = new VolumeParameter(1);

    @serializable(VolumeParameter)
    readonly qualityLevel: VolumeParameter = new VolumeParameter(0);

    @serializable(VolumeParameter)
    readonly predicationMode: VolumeParameter = new VolumeParameter(0);

    @serializable()
    readonly surpressMsaa: boolean = true;

    onCreateEffect(): EffectProviderResult {
        const effect = new MODULES.POSTPROCESSING.MODULE.SMAAEffect({
            preset: this.getPreset(this.qualityLevel.value),
            edgeDetectionMode: this.getEdgeDetectionMode(this.edgeDetectionMode.value),
            predicationMode: this.getPredicationMode(this.predicationMode.value)
        });

        this.edgeDetectionMode.onValueChanged = (newValue) => {
            effect.edgeDetectionMaterial.edgeDetectionMode = this.getEdgeDetectionMode(newValue);
        };

        this.qualityLevel.onValueChanged = (newValue) => {
            effect.applyPreset(this.getPreset(newValue));
        }

        this.predicationMode.onValueChanged = (newValue) => {
            effect.edgeDetectionMaterial.predicationMode = this.getPredicationMode(newValue);
        };

        return effect;
    }

    update(): void {
        if(this.surpressMsaa && this.context.composer && this.context.composer.multisampling > 0) {
            this.context.composer.multisampling = 0;
        }
    }

    getEdgeDetectionMode(value: EdgeDetectionMode): _EdgeDetectionMode {
        switch(value)
        {
            default:
            case EdgeDetectionMode.COLOR:
                return MODULES.POSTPROCESSING.MODULE.EdgeDetectionMode.COLOR;

            case EdgeDetectionMode.DEPTH:
                return MODULES.POSTPROCESSING.MODULE.EdgeDetectionMode.DEPTH;

            case EdgeDetectionMode.LUMA:
                return MODULES.POSTPROCESSING.MODULE.EdgeDetectionMode.LUMA;                
        }
    }

    getPreset(value: QualityLevel): SMAAPreset {
        switch(value)
        {
            default:
            case QualityLevel.LOW:
                return MODULES.POSTPROCESSING.MODULE.SMAAPreset.LOW;

            case QualityLevel.MEDIUM:
                return MODULES.POSTPROCESSING.MODULE.SMAAPreset.MEDIUM;

            case QualityLevel.HIGH:
                return MODULES.POSTPROCESSING.MODULE.SMAAPreset.HIGH;

            case QualityLevel.ULTRA:
                return MODULES.POSTPROCESSING.MODULE.SMAAPreset.ULTRA;
        }
    }

    getPredicationMode(value: PredicationMode): _PredicationMode {
        switch(value) {
            default:
            case PredicationMode.DISABLED:
                return _PredicationMode.DISABLED;
            
            case PredicationMode.DEPTH:
                return _PredicationMode.DEPTH;
        }
    }
}

registerCustomEffectType("SMAAEffect", SMAAEffect)

Would be great to just use MSAA all the time but at least for fullscreen scenarios it’s too expensive right now.