Neural Network Application Methods Used to Evaluate and Increase Photorealism of Virtual Reality
Abstract
Neural Network Application Methods Used to Evaluate and Increase Photorealism of Virtual Reality
Incoming article date: 26.05.2019The article focuses on the possibility of using neural networks to increase the level of the graphic component of virtual reality. An image quality evaluation method using three criteria is examined: photorealism of image macrostructure, photorealism of image microstructure, as well as the level of its distortion. Its implementation relies on the VGG neural network, trained in a variety of images, evaluated by the presented criteria. After metric realization is complete, a method is proposed that has to do with automatic selection of parameters and contents of the virtual scene with metric maximization in the images obtained from different angles. Next, a method is considered and evaluated, where rendering algorithm is replaced by a neural network. At which point, methods to improve the quality of the final image are suggested. Based on the results of the experiments, Wasserstein generative adversarial networks (WGAN), distinguished by the highest level of image quality, are singled out. Further on an option is proposed to improve this approach in terms of speed and quality of the result that consists of breaking up one large network into an ensemble of smaller ones. It should be emphasized that each network of the ensemble processes only part of the image relating to one type of segment. Next, it is prudent to consider the possibility of applying this algorithm in real time, to which end optimization of neural networks is conducted to the detriment of quality, but with a significant increase in execution speed.
Keywords: Image quality enhancement, image quality evaluation metrics, neural networks, virtual reality, synthetic data, GAN, WGAN, photorealistic synthetic images