SUMM

AI relighting for 2D photographs is introduced as a novel technique, allowing changes to light sources after the photo is taken.
Traditional tools like Photoshop cannot manipulate light direction in 2D images; only basic adjustments (like contrast) are possible.
In 3D modeling programs (e.g., Blender), lighting can be changed easily, but this hasn’t translated to 2D photography—until now.
New research promises the ability to change lighting conditions (e.g., time of day) in an ordinary photo, creating very realistic results.

The process begins by "delighting" the image—removing existing lighting using previously published methods.
The next step is to 3D-ify the image: convert the 2D photo into a rough 3D scene using AI techniques.
Initial 3D reconstructions are rough and contain errors or holes, which are apparent when rendered in new lighting environments.
The novel step involves beautifying the rough 3D render into a realistic photo through a neural renderer.
A neural network is trained to convert rough renders into photorealistic images, requiring pairs of photos and matching rough 3D renders.

The challenge is obtaining rough 3D renderings that match the original lighting of photos.
The researchers iteratively add and adjust artificial light sources in the 3D scene to minimize difference from the original, repeating this across thousands of photos.
This approach enables the training of a robust neural renderer, capable of relighting images believably.
With this innovation, one can now easily add or remove lights, change day to night, and even use spotlights or animated projectors with cast shadows.

The entire process is extremely fast, taking only about three seconds per photo—two seconds for preprocessing and less than one second for relighting.
This paper and technology, despite its capability, has not received significant public attention.
Limitations include blocky artifacts when lighting moves, resolution of 3D geometry still lacking, and problems with complex materials (like skin or specular highlights).
Placing lights behind the scene or in unlikely places can cause visual errors.
The current method struggles with complex images but shows significant improvement over previous techniques.

This kind of AI relighting signals a fundamental shift: from static photos as mere memories to dynamic, editable worlds.
Artists gain the ability to direct the reality of a photo even after it’s taken.
The presenter praises the paper and the team’s presentation, calling the innovation brilliant.
An unrelated mention is made of experimenting with OpenAI’s large GPT model through Lambda GPU cloud, highlighting its speed and accessibility for a low cost.

Forgotten AI Research Solved The Problem Photoshop Never Could!