Software for converting two-dimensional images into three-dimensional models

Purpose of reseach. Development of a software package for the automatic creation of photorealistic threedimensional models of faces based on a single input image, which will provide a high degree of detail and realism of models, as well as ease of use,

Methods. The study uses an integrated approach to create photorealistic three-dimensional models of faces from two-dimensional images based on reverse rendering methods and cascading convolutional neural networks (CNN). The main element is a three-dimensional transformable model (3DMM), which describes the geometry and albedo of a face through linear combinations of principal component bases (PCA). To match the 3D geometry and the 2D image, a weak perspective projection is used, taking into account Euler angles and lighting conditions. Optimization of the objective function using the Gauss-Newton method minimizes the differences between the input and rendered images, and correction of depth and facial details is achieved through the adaptation of 3D graphics. Linear albedo interpolation clarifies the details of the model in key areas, which contributes to the creation of high-quality and realistic 3D models of faces.

Results. This article successfully implements a software package capable of generating photorealistic threedimensional models of faces from one-dimensional images using reverse rendering and cascading convolutional neural networks. The experiments have confirmed the algorithm's ability to perceive important facial characteristics and create opportunities for further applications in the fields of computer graphics, animation and virtual interfaces.

Conclusion. The results obtained indicate the high efficiency of the developed algorithm for generating photorealistic three-dimensional models of faces from two-dimensional images. In addition, the results confirm that the use of reverse rendering methods in conjunction with cascading convolutional neural networks allows for significant improvements in visualization quality.

1. Lee H., Ranganath R., Ng A.Y. Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations. Grosse. Proceedings of the 26th Annual International Conference on Machine Learning. 2009. P. 34-45.

2. Bengio Y. Learning deep architectures for AI. Foundations and Trends in Machine Learning. 2009. P. 245-257.

3. Ignatenko A., Konushin A. Framework for Depth Image-Based Modeling and Rendering. Graphicon-2003 Proceedings. Moscow; 2003. 246 p.

4. Vasilenko B. A. Spline functions: theory, algorithms, programs. Novosibirsk: Nauka; 1983. 215 p. (In Russ.).

5. Solovyova A. Algorithm for modification of a typical three-dimensional portrait based on specified photographic images. In: Trudy XX mezhdunarodnoi konferentsii po komp'yuternoi grafike i mashinnomu zreniyu = Proceedings of the XX International Conference on Computer Graphics and Machine Vision. Moscow; 2010. P. 346-347. (In Russ.).

6. Zhao H., Osher S. Visualization, analysis and shape reconstruction of unorganized data sets. In: Geometric Level Set Methods in Imaging and Vision and Graphics. SpringerVerlag, 2002. 256 p.

7. Zotkina A.A. Analysis of machine learning algorithms used in the classification of images published by users of social networks. Sovremennye informatsionnye tekhnologii = Modern information technologies. 2023; (38): 38-40. (In Russ.).

8. Zotkina A.A. Image recognition using convolutional neural networks. Sovremennye informatsionnye tekhnologii= Modern information technologies. 2023; (38): 60-63. (In Russ.).

9. Zotkina A.A., Martyshkin A.I., Tkachenko A.V. Features of convolutional neural networks: architecture and application. In: Sovremennye metody i sredstva obrabotki prostranstvennovremennykh signalov. Sbornik statei XX Vserossiiskoi nauchno-tekhnicheskoi konferentsii = Modern methods and means of processing spatiotemporal signals. Collection of articles of the XX AllRussian Scientific and Technical Conference. Penza; 2023. P. 32-35. (In Russ.).

10. Blanz V., Vetter T. Face recognition is based on fitting a 3-D morphable model. IEEE Trans Pattern Anal Mach Intell. 2003; 25(9):1063-1074.

11. Richardson E., Sela M., OR-EL R., Kimmel R. Learning detailed face reconstruction from a single image. In: IEEE conference on computer vision and pattern recognition (CVPR). Honolulu, HI; 2017. P. 5553-5562.

12. Blanz V., Vetter T.A. Morphable model for the synthesis of 3-D faces. In: 26th annual conference on Computer graphics and interactive techniques (SIGGRAPH ’99). ACM Press/AddisonWesley Publishing Co., USA; 1999. P. 5553-5562.

13. Heo J.Three-dimensional generic elastic models for two-dimensionalpose synthesis and face recognition. Proquest, Umi Dissertation Publishing; 2011. 154 p.

14. Korikov A.M., Tungusova A.V. Neural network technologies for image classification. 21st Int. Symp. Atmos. Ocean Opt. Atmos. Phys. 2015; 9680: 426–429.

15. Krizhevsky A., Sutskever I., Hinton G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012; 25.

16. Zhao W., Chellappa R., Phillips P.J., Rosenfeld A. Face recognition: A literature survey. ACM Comput. Surv. 2003; 35(4): 399 – 458.

17. Zhang L., Lin L., Liang X., He K. Is faster R-CNN doing well for pedestrian detection? Eur. Conf. Comput. vision. Springer, Cham; 2016. P. 443–457.

18. Coelho L. P., Richart V. Building machine learning systems in Python. Moscow: DMK Press; 2016. 302 p. (In Russ.). Available at: https://e.lanbook.com/book/82818

19. Flach P. Machine learning. The science and art of building algorithms that extract knowledge from data. Moscow: DMK Press; 2015. 400 p. (In Russ.). Available at: https://e.lanbook.com/book/69955

20. PyTorch. Available at: https://pytorch.org/

21. Numpy. Available at: https://numpy.org/

22. How to install dlib library for Python in Windows 10. Available at: https://medium.com/analytics-vidhya/how-to-install-dlib-library-for-python-in-windows-10-57348ba1117f23 Opencv. Available at: https://opencv.org/