I’m trying to change an image source at runtime, based on some audio levels. It’s meant to act as a video playhead controller. I got everything working except the sprite doesn’t update. I have a reference to the sprite, and I’m setting it on the image component, but the visuals never update.
import { AudioSource, Behaviour, serializable, Sprite, Image } from "@needle-tools/engine"
import { AudioAnalyser, Audio } from "three"
function lerp(start, end, amt) {
return (1 - amt) * start + amt * end
}
export class AudioViz extends Behaviour {
@serializable(Array<Sprite>)
frames: Sprite[] = []
@serializable(Image)
image?: Image
@serializable(AudioSource)
audio?: AudioSource
private analyser?: AudioAnalyser
private smoothLevel: number = 0
private didInit: boolean = false
initAnalyzer() {
if (!this.audio) return
this.didInit = true
const sound = this.audio.Sound as any as Audio
this.analyser = new AudioAnalyser(sound, 64) // fft size
}
update(): void {
if (this.audio?.isPlaying && !this.didInit) this.initAnalyzer()
const freq = this.analyser?.getAverageFrequency() || 0
this.smoothLevel = lerp(this.smoothLevel, freq / 255, .5)
if(this.image){
this.image.sprite = this.frames[Math.floor(this.smoothLevel * 150) % 20]
}
}
}
I think we never did that before so it doesnt automatically update the underlying objects. I’ll check. One thing i notice that is wrong/doenst work like that I think is the @serializable(Array<Sprite>) - it should be @serializable(Sprite) (declaring the type - not the array) - it will surely not be enough to make it work tho
One more thing I noticed is that there’s no built in event for when an audio component starts playing, so I had to do this state checking in the update handler. Would be a nice quality of life addition to emit an event for that.
