Computing and Mathematical Sciences @ Caltech
We consider solving ill-posed imaging inverse problems without access to an image prior or ground truth examples. An overarching challenge in these inverse problems is that an infinite number of images, including many that are implausible, are consistent with the observed measurements. Thus, image priors are required to reduce the space of possible solutions to more desirable reconstructions. However, in many applications it is difficult or potentially impossible to obtain example images to construct an image prior. Hence inaccurate priors are often used, which inevitably result in biased solutions. Rather than solving an inverse problem using priors that encode the spatial structure of any one image, we propose to solve a set of inverse problems jointly by incorporating prior constraints on the collective structure of the underlying images. The key assumption of our work is that the underlying images we aim to reconstruct share common, low-dimensional structure. We show that such a set of inverse problems can be solved simultaneously without the use of a spatial image prior by instead inferring a shared image generator with a low-dimensional latent space. The parameters of the generator and latent embeddings are found by maximizing a proxy for the Evidence Lower Bound (ELBO). Once identified, the generator and latent embeddings can be combined to provide reconstructed images for each inverse problem. The framework we propose can handle general forward model corruptions, and we show that measurements derived from only a small number of ground-truth images (⩽ 150) are sufficient for “prior-free” image reconstruction. We demonstrate our approach on a variety of convex and non-convex inverse problems, ranging from denoising, phase retrieval, and black hole video reconstruction.
Journal paper with supplement [TCI paper pdf][arxiv link]
ICASSP paper [ICASSP pdf]
Black hole imaging paper [pdf]
Code [Github]
Gao, A.F., Leong, O., Sun, H. and Bouman, K.L. "Image Reconstruction without Explicit Priors." ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023.
bibtex [show]
Leong, Oscar, et al. "Discovering Structure From Corruption for Unsupervised Image Reconstruction." IEEE Transactions on Computational Imaging 9 (2023): 992-1005.
bibtex [show]
Lin, Yvette Y., Angela F. Gao, and Katherine L. Bouman. "Imaging An Evolving Black Hole By Leveraging Shared Structure." ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024.
bibtex [show]
We seek to perform unsupervised ill-posed image reconstruction without access to an image prior. We define an image generation model (IGM) to be a model that captures the distribution of spatial structure within images. We design a criterion to: 1. select an IGM from a set of IGMs to best reconstruct an image from measurements, and 2. infer the best IGM to reconstruct a set of images from noisy measurements.
We show that solving such ill-posed imaging inverse problems is possible when there is shared structure across the clean underlying images.
Our method is able to remove much of the corruptions in these images and outperform baselines designed to solve various imaging inverse problems.
We use our method to perform black hole video reconstruction using 60 frames.
We use our method to perform black hole video reconstruction using 60 frames where we assume realistic atmospheric errors. Thus, we are solving a black hole phase retrieval problem using closure phases and visibility amplitudes as our measurements.
