Person re-identification in video surveillance systems by feature replacement of occluded parts of human figures

Abstract

Тhe algorithm for re-identifying people in intelligent video surveillance systems is proposed. It is based on the construction of a compound neural network descriptor and replacing features of occluded parts of a human figure. Composite descriptors are generated for all images in a gallery and recorded in a table. They characterize the global and local features of each person, considering his individual parts’ visibility. Detection and selection of areas of interest for the formation of local descriptors is carried out based on detecting key points of the human body. If a person is partially occluded by other people or objects, then the corresponding region is classified as invisible, and the compound descriptor of respective component will be invalid and equal to zero. For images whose feature vector has zero components, the feature table is ranked according to the cosine similarity metric for each visible local fragment. Based on the feature table rankings, the k-nearest neighbors are determined and the k1-best are selected from them. The corresponding k1-nearest neighbors’ component average value of the feature vector is used to replace the zero descriptor components. The feature table is then updated for the generated vectors and ranking is performed according to the query using the cosine similarity metric. ResNet-50 and DenseNet-121 were used as backbone CNNs for feature extraction, and testing was performed using Market-1501, DukeMTMC-ReID, Occluded-Duke, MSMT17, and PolReID1077 datasets.