Masked Autoencoders

We explore data-independent masking strategies for Masked AutoEncoders (MAE), showing that carefully designed structured noise masks can match or improve upon learned/data-dependent masking, while being simpler and more efficient. See our [Project …