Okay, I just tested it. The ctx.renderer.clearDepth() doesnt do anything alone. So I played around with the buffer clearing, and found out, that when i call clearColor(), it resembles this three.js examples (so the model is like flickering, but there is no depth overlay, like here VR on pico does not clear buffer), and if i run either clearDepth() and clearColor() or clear(), it works fine. So for now, I would say the fix is add this snippet to the main.ts:
onBeforeRender(ctx => {
if(ctx.isInXR) ctx.renderer.clear();
})
But I dont understand why clearing of the color buffer resolves most of the problem.