Two pairs of researchers from Cornell University and Adobe have teamed up and developed a “Deep Photo Style Transfer” algorithm that can automatically apply the style (read: color and lighting) of one photo to another. The early results are incredibly impressive and promising.
The software is an expansion on the tech used to transfer painting styles like Monet or Van Gogh to a photograph like the app Prisma. But instead of a painting, this program uses other photographs for reference.
“This paper introduces a deep-learning approach to photographic style transfer that handles a large variety of image content while faithfully transferring the reference style,” says the rather technical abstract of the Deep Photo Style Transfer paper.
Put more plainly: when you put in two photographs, the neural network-powered program analyzes the color and quality of light in the reference photo, and pastes that photo’s characteristics onto the second. This includes things like weather, season, and time of day—theoretically, a winter’s day can be turned into summer, or a cloudy day into a glorious sunrise.
The team’s early examples show the program in action. So this original photo:
Plus this reference photo:
Equals this final photo:
It’s important to note that the software does not alter the structure of the photo in any way, so there’s no risk of distorting the lines, edges or perspective. The entire focus is on mimicking the color and light in order to copy the “look” or “style” of a reference photograph onto a new shot.
Since this is a lot easier said than done, the program has to intelligently compensate for differences between the donor and receiving image. If there is less sky visible in the receiving image, it will detect this difference and not cause the sky to spill over into the rest of the original shot, for example.
The software even attempts…