Screens of CCTV footage

AI technique provides realistic reconstruction of pixelated images

Image credit: Dreamstime

The creation of machine learning software, which can transform a blurry photo into a realistic, high-quality image, could allow for old photographs and films to be enhanced and pixelated images to be restored.

The software, named EnhanceNet, was developed by researchers at the Max Planck Institute for Intelligent Systems in Tübingen, Germany, and can vastly enhance the quality of blurry or heavily pixelated images.

Already, there are many tools available which are capable of automatically sharpening images. However, these tools typically use pixel-wise reconstruction measures, such as peak signal-to-noise ratio. These approaches tend to produce “over-smoothed” results, which look unrealistic to the human eye.

Instead, EnhanceNet uses texture analysis powered by machine learning, which allows it to recreate false, realistic textures which are appropriate for the image. It is even capable of reproducing the depth of field of the camera from the original photograph. Although not pixel perfect, these reproductions appear realistic.

“We propose a novel application of automated texture synthesis in combination with a perceptual loss focusing on creating realistic textures rather than optimising for a pixel-accurate reproduction of [original] images during training,” the researchers wrote in their paper detailing their work.

Enhanced image of a grasshopper

Mehdi SM Sajjadi, Bernhard Scholkopf, Michael Hirsch

Image credit: Mehdi SM Sajjadi, Bernhard Scholkopf, Michael Hirsch

Enhancing blurry image of an eagle

Mehdi SM Sajjadi, Bernhard Scholkopf, Michael Hirsch

Image credit: Mehdi SM Sajjadi, Bernhard Scholkopf, Michael Hirsch

Despite introducing false textures and creating other details, EnhanceNet is capable of reproducing the original photographs with startlingly high accuracy.

EnhanceNet uses adversarial training to produce its realistic results. During this type of training, an artificial neural network – a computer system loosely inspired by the behaviour of brains – generates results based on millions of low-quality images while another network analyses the quality of the results by attempting to recognise objects, such as a face or an animal, in the image. This feedback allows for the network to be refined.

The researchers told FastCoDesign that this efficient technique could eventually be used in image-editing software such as Photoshop, could be incorporated into operating systems to restore the quality of images during zooming, and could even be used to reverse the pixilation of obscured photographs.

While this could have useful security applications, such as allowing police officers to acquire high-quality images of suspicious vehicles, such a method becoming widespread could also have privacy implications, potentially rendering pixilation for anonymity reversible.

Recent articles

Info Message

Our sites use cookies to support some functionality, and to collect anonymous user data.

Learn more about IET cookies and how to control them