Efficient Maritime Vessel Crowded Video Summarization Using Machine Learning and Cross-Modal Approach

Abstract

The exponential growth of surveillance footage, particularly in densely crowded environments, poses substantial challenges for efficient maritime vessel video analysis and summarization. Conventional methods often struggle to manage the inherent complexity and redundancy within such data. Addressing these challenges, this study presents a novel behavior-based crowded video summarization framework that synergistically integrates machine learning and cross-modal techniques to extract meaningful insights. Central to the approach is a contrastive learning network that effectively identifies informative frames, enabling the generation of concise video summaries enriched with textual descriptions. Feature extraction is powered by a Generative Adversarial Network (GAN), complemented by knowledge distillation for optimal keyframe selection. To further improve event recognition and summarization accuracy, a Dual Graph Neural Network (DGNN) is employed. Additionally, a hybrid optimization strategy combining Lion-Deer Hunting Optimization with the Jaya algorithm ensures precise event capture. Experimental results demonstrate the framework’s superior performance, achieving a precision of 99.03%, recall of 99.08%, and accuracy of 99.96%. These outcomes validate the method’s capability to produce compact and accurate summaries that effectively highlight critical human behavior patterns in complex, maritime vessel crowded video scenarios.