Few-shot Image Generation with Elastic Weight Consolidation

Yijun Li
Richard Zhang
Jingwan Lu
Eli Shechtman

NeurIPS 2020



Few-shot image generation seeks to generate more data of a given domain, with only few available training examples. As it is unreasonable to expect to fully infer the distribution from just a few observations (e.g., emojis), we seek to leverage a large, related source domain as pretraining (e.g., human faces). Thus, we wish to preserve the diversity of the source domain, while adapting to the appearance of the target. We adapt a pretrained model, without introducing any additional parameters, to the few examples of the target domain. Crucially, we regularize the changes of the weights during this adaptation, in order to best preserve the “information” of the source dataset, while fitting the target. We demonstrate the effectiveness of our algorithm by generating high-quality results of different target domains, including those with extremely few examples (e.g., ≤10). We also analyze the performance of our method with respect to some important factors, such as the number of examples and the dissimilarity between the source and target domain.



Visual comparisons of different methods for few-shot generation (FFHQ -> Emoji). Left: 10-shot training examples; Right: 10 generated results by each method.

This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project. The code can be found here.