BigColor:
Colorization using a Generative Color Prior for Natural Images

Abstract

For realistic and vivid colorization, generative priors have recently been exploited. However, such generative priors often fail for in-the-wild complex images due to their limited representation space. In this paper, we propose BigColor, a novel colorization approach that provides vivid colorization for diverse in-the-wild images with complex structures. While previous generative priors are trained to synthesize both image structures and colors, we learn a generative color prior to focus on color synthesis given the spatial structure of an image. In this way, we reduce the burden of synthesizing image structures from the generative prior and expand its representation space to cover diverse images. To this end, we propose a BigGAN-inspired encoder-generator network that uses a spatial feature map instead of a spatially-flattened BigGAN latent code, resulting in an enlarged representation space. Our method enables robust colorization for diverse inputs in a single forward pass, supports arbitrary input resolutions, and provides multi-modal colorization results. We demonstrate that BigColor significantly outperforms existing methods especially on in-the-wild images with complex structures.

Framework

BigColor consists of a class-conditioned encoder and fine layers of pretrained BigGAN generator. The key idea is to use a spatial feature map as latent code instead of spatially-flattened latent code of BigGAN, which effectively transfers a structure information of an input image. Then, jointly training the encoder-generator model enables us to learn the generative color prior with an enlarged representation space. Additionally, The randomlysampled enables generating multi-model solutions. The class label used in a feedforward can be estimated by an off-the-shelf classifier or be specified by a user.

Results

Figure 1: Black-and-white photography colorization with arbitrary resolution.

Figure 2: Multi-modal solutions from a graysacale image.

Figure 3: Uncurated colorization results.

Citation

 @inproceedings{Kim2022BigColor,
  title     = {BigColor: Colorization using a Generative Color Prior for Natural Images},
  author    = {Geonung Kim,Kyoungkook Kang,Seongtae Kim,Hwayoon Lee,Sehoon Kim,Jonghyun Kim,Seung-Hwan Baek,Sunghyun Cho},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2022}}