The approximation of the game’s San Andreas to the real-life Los Angeles and Southern California makes the Game Theft Auto V more special. But Intel Labs introduces a new machine learning project called “Enhancing Photorealism Enhancement” that intends to push the game towards photorealism (via Gizmodo).
Researchers Stephan R. Richter, Hassan Abu Alhaija, and Vladlen Kolten worked on the game and produced a surprising result: a visual look with unmistakable similarities to the kinds of photos one might take through the smudged front window of the car. It’s similar to the situation where you’re looking out at the real street from an actual dashboard, even when it’s a virtual world.
The Intel researchers explain the working of this enhancement with the Cityscapes Dataset that was fed to the neural networks. This dataset is built mainly from photographs of German streets — filled in a lot of real-life details. It’s dimmer than the earlier one and from a different angle, but it almost captures a somewhat smoother and more interactive version of scrolling through Google Maps’ Street View. In other words, the Intel-processed image is dull. It turns a once brightly lit scene with some exciting splashes of color into a dull, gray, overcast environment. But it also looks more realistic to the eyes.
As the researchers say, this enhancement goes beyond what other photorealistic conversion processes are capable of, as they have also integrated geometric information from GTA V itself. This information is represented in the form of “G-buffers,” which can include data like the distance between objects and the camera, and the quality of textures, like the cars’ glossiness, etc.
While one might not see an official “photorealism update” roll out to GTA V very soon, but it’s possible that one might have already played a game or watched a video that’s benefited from another kind of ML — AI upscaling. The process of using ML smarts to blow up graphics to higher resolutions doesn’t show up everywhere. Still, it has been featured in Nvidia’s Shield TV and in several other mod projects aimed at upgrading the graphics of older games. In those cases, a neural network makes predictions to fill in missing pixels of detail from a lower resolution game, movie, or TV show to reach those higher resolutions.
The goal wasn’t to create a “realism” filter that gets slapped on every title. This work is a concrete step towards achieving better photorealism overall.
Paper: https://intel-isl.github.io/PhotorealismEnhancement/
Video Paper: https://www.youtube.com/watch?v=P1IcaBn3ej0
Shilpi is a Contributor to Marktechpost.com. She is currently pursuing her third year of B.Tech in computer science and engineering from IIT Bhubaneswar. She has a keen interest in exploring latest technologies. She likes to write about different domains and learn about their real life applications.
