Generative Representation Learning for Scalable Latent Tokenization

The short version

The problem

Image generators often first turn an image into a set of compact features, or tokens. Features trained to recognize what matters for one task can throw away other details, making it harder for a generator to recreate the full image—and training a separate tokenizer is only worthwhile if it saves compute overall.

The idea

RePR teaches image-patch features to record how much one visible part helps predict another. It estimates that help by comparing a generative model’s predictions with and without the extra context, then trains the features to reproduce the difference. For example, if seeing one region makes a neighboring patch easier to predict, their features should reflect that shared information; the method also builds up estimates from smaller prediction steps and can emphasize information that survives stronger corruption.

What it got

RePR’s image-generation system achieved a Fréchet Inception Distance (FID, a measure of generated-image quality where lower is better) of 2.18 on ImageNet, compared with 2.36 for direct pixel generation at roughly equal training compute. Its results also show that the learned representations improve predictably as training compute increases.