Facial keypoints detection using capsule neural networks

The problem of detecting key points of the face is investigated. This problem is quite relevant and important. The existing approaches of solving this problem, which are usually divided into parametric and nonparametric methods, are considered. As a result of the study, it was concluded that, nowadays, the most qualitative results are demonstrated by approaches based on deep learning methods. Two solutions are proposed: a capsule network with dynamic routing and a deep capsule network. The data for the experiments are 10,000 generated faces taken from Kaggle, marked up using MediaPipe. A method of using capsule architectures in neural networks to solve the problem of detecting key points of the face is proposed. The method includes the use of segmentation based on the key points of the face recognized using MediaPipe. Delaunay triangulation was used to build the face mesh. The architecture of a deep capsule network considering semantic segmentation was proposed. Based on the marked-up data, experiments on the detection of key points using the developed capsule neural networks were performed. According to the test results, the loss function reached values in range 2.50–2.90, the accuracy reached values in range 0.87–0.9. The proposed architecture can be used in technologies for comparing the geometry of the face grid of a real person with the geometry of the face grid of a three-dimensional model as well as in further studies of capsule neural networks by researchers in the field of image processing and analysis.