GenLit: Reformulating Single-Image Relighting as Video Generation

Shrisha Bharadwaj*, Haiwen Feng*, Giorgio Becherini, Victoria Fernandez Abrevaya, Michael J. Black

Max Planck Institute for Intelligent Systems, Tübingen, Germany

*Equal Contribution, listed alphabetically

arXiv Coming soon!

GenLit is a new framework for single image relighting, where the task is reformulated as image-to-video generation by keeping the scene and the object in the image static while generating the lighting changes (motion) by controlling the position of the light source (point light).


Object-Level: Generalization to Real Images

Input | Ours | Reference

See more comparisons

Object-Level: Comparisons with State-of-the-Art

See more comparisons

Input

WS-SIR

Neural Gaffer

IC-Light

DelightNet

Ours

Ground Truth

Scene-Level: Generalization to Real Images

Input | Ours | Reference

See more comparisons

Scene-Level: MIT Multi-Illumination Dataset

See more comparisons

Input

Latent-Intrinsics

Ours

GT

Acknowledgements

We thank Peter Kulits for discussions, proofreading, and Suraj Bhor for helping us with the baselines. We thank Nikos Athanasiou for providing feedback. We are very grateful for the support provided by Tsvetelina Alexiadis, Florentin Doll, Markus Höschle, Arina Kuznetcova, Tomasz Niewiadomski, Taylor Obersat and Tithi Rakshit with the data and Rick Akkerman for his help with code. We thank Prerana Achar, Radek Daněček, Shashank Tripathi and Anastasios Yiannakidis for their support with visualizations. We additionally thank Peter Kulits for his effective ninjutsu skills. Finally, we thank Pramod Rao for fruitful discussions and constant support.