DEEP FUSION: A DEEP LEARNING FRAMEWORK FOR THE FUSION OF HETEROGENEOUS SENSORY DATA

Abstract

Deep learning has revolutionized the integration of heterogeneous sensory data,facilitating the extraction of valuable insights from multiple sources. This study introduced DeepFusion, a deep learning-based framework aimed at enhancing multi-sensor data fusion by analyzing cross-sensor correlations and adaptively assigning weights according to measurement quality. The primary objectives were to develop an efficient sensor fusion model, evaluate its effectiveness in human activity recognition using real-world datasets, and compare its performance against existing approaches. While multi-sensor fusion improves accuracy, resilience to noise, and feature diversity, it also poses challenges such as increased computational demands and the need for data synchronization. To assess DeepFusion, two testbeds were constructed utilizing commercially available sensors, including smartphones, smartwatches, Shimmer sensors, WiFi, and acoustic sensors. Experimental evaluations revealed that DeepFusion outperformed leading methods in human activity recognition. The dataset encompassed various human activities in a device-free setting, such as typing, writing, and walking. Despite classification challenges observed in the confusion matrix, DeepFusion achieved the highest accuracy (0.908) on the CSI dataset, exceeding the performance of DeepSense (0.860), SR+WC (0.865), SR+Avg (0.833) and SVM (0.520), demonstrating its superiority in multi-sensor data integration.