Visual intelligence-driven hybrid feature learning with efficient dual-stage recurrent attention network for human activity recognition

초록

Over the past few years human activity recognition (HAR) achieved remarkable attention among researchers, video surveillance, business communities, industrial applications, and many other real-world entities. However, existing video surveillance systems of HAR mainly focuses on achieving high recognition performance, and neglect the system computational complexity. Thus, current methods struggle to process visual data using resource-constrained devices. Therefore, in this paper, we consider both computational complexity and recognition accuracy. The proposed study make few major contributions from the following aspects. We first, retrieved frames sequence of continuous streaming videos using temporal stride. Evidently, this step provide medium frames per second (mfps) which stabilize computational complexity of the system. We then, employed two different pre-trained architectures aiming to achieve enrich hidden patterns of human activities. Moreover, this strategy helps reduce computational complexity of the downstream architecture. Following this, those representations fuses in order to obtain a comprehensive detail of human appearances. Second, the previously fused features are then passed to a novel architecture, namely dual-stage recurrent attention network (DSRAN). The proposed DSRAN learn spatio-temporal dynamics in forward and backward direction at each timestep t. Consequently, these strategies of our proposed approach greatly helps in learning spatio-temporal patterns of human activities. We conducted extensive simulations using four publicly available datasets of human activities, including UCF101, HMDB51, UCF50, and YouTube11, achieving accuracies of 97.59%, 79.82%, 98.52%, and 99.03%, respectively. The comprehensive results of our proposed method have shown dominance performance against existing state-of-the-art (SOTA) methods.

키워드