Video Understanding

Structured-Noise Masked Modeling for Video, Audio and Beyond

We filter white noise into different color noise distributions to build structured masks for masked video and audio modeling: Green3D for video and Regularized Blue Noise for audio, improving over random masking at no additional computational cost.