HierFuse: Hierarchical Gated Fusion Network for Parkinson’s Disease Screening Based on Hand Tremor Characteristics

LiHao Zhao

doi:10.63313/JCSFT.9052

Authors

LiHao Zhao College of Computer Science and Technology, Qingdao University, Qingdao 266071, China Author

DOI:

https://doi.org/10.63313/JCSFT.9052

Keywords:

Multimodal, Convolutional Neural Network, Parkinson's Disease

Abstract

Parkinson's disease (PD) impairs motor control, making handwriting analysis a valuable non-invasive biomarker. This paper presents HierFuse, a hierarchical gated multimodal fusion network for PD screening on the NewHandPD dataset, which provides paired images and sensor signals from drawing tasks. HierFuse integrates temporal 1D and spatial 2D convolutional branches through cross-scale alignment, shared attentive compression, and progressive gated fusion with hidden propagation. Five-fold cross-validation shows state-of-the-art performance on Spiral (acc 0.912, F1 0.908) and Meander tasks (acc 0.929, F1 0.927), outperforming baselines. Ablations validate component contributions, with hierarchical fusion and hidden propagation most critical, while gating shows task-specific effects. Results demonstrate that hierarchical multimodal fusion effectively leverages handwriting dynamics for accurate PD detection.

References

[1] Jankovic J. Parkinson’s disease: clinical features and diagnosis[J]. Journal of neurology, neurosurgery & psychiatry, 2008, 79(4): 368-376.

[2] Hughes A J, Daniel S E, Kilford L, et al. Accuracy of clinical diagnosis of idiopathic Parkinson's disease: a clinico-pathological study of 100 cases[J]. Journal of neurology, neurosurgery & psychiatry, 1992, 55(3): 181-184.

[3] Li J, Jin M, Wang L, et al. MDS clinical diagnostic criteria for Parkinson’s disease in China[J]. Journal of neurology, 2017, 264(3): 476-481.

[4] Schrag A, Ben-Shlomo Y B, Quinn N. How valid is the clinical diagnosis of Parkinson's disease in the community?[J]. Journal of Neurology, Neurosurgery & Psychiatry, 2002, 73(5): 529-534.

[5] Drotár P, Mekyska J, Rektorová I, et al. Evaluation of handwriting kinematics and pressure for differential diagnosis of Parkinson's disease[J]. Artificial intelligence in Medicine, 2016, 67: 39-46.

[6] Pereira C R, Weber S A T, Hook C, et al. Deep learning-aided Parkinson's disease diagnosis from handwritten dynamics[C]//2016 29th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI). Ieee, 2016: 340-346.

[7] Pereira C R, Pereira D R, Rosa G H, et al. Handwritten dynamics assessment through convolutional neural networks: An application to Parkinson's disease identification[J]. Artificial intelligence in medicine, 2018, 87: 67-77.

[8] San Luciano M, Wang C, Ortega R A, et al. Digitized spiral drawing: A possible biomarker for early Parkinson’s disease[J]. PloS one, 2016, 11(10): e0162799.

[9] Diaz M, Moetesum M, Siddiqi I, et al. Sequence-based dynamic handwriting analysis for Parkinson’s disease detection with one-dimensional convolutions and BiGRUs[J]. Expert Systems with Applications, 2021, 168: 114405.

[10] Skaramagkas V, Pentari A, Kefalopoulou Z, et al. Multi-modal deep learning diagnosis of Parkinson’s disease—a systematic review[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 2399-2423.

[11] Zhao A, Wu H, Chen M, et al. A Spatio‐Temporal Siamese Neural Network for Multimodal Handwriting Abnormality Screening of Parkinson’s Disease[J]. International Journal of Intelligent Systems, 2023, 2023(1): 9921809.

[12] Pereira C R, Pereira D R, Silva F A, et al. A new computer vision-based approach to aid the diagnosis of Parkinson's disease[J]. Computer methods and programs in biomedicine, 2016, 136: 79-88.

[13] Folador J P, Santos M C S, Luiz L M D, et al. On the use of histograms of oriented gradients for tremor detection from sinusoidal and spiral handwritten drawings of people with Parkinson’s disease[J]. Medical & biological engineering & computing, 2021, 59(1): 195-214.

[14] Huang Y, Chaturvedi K, Nayan A A, et al. Early Parkinson’s disease diagnosis through hand-drawn spiral and wave analysis using deep learning techniques[J]. Information, 2024, 15(4): 220.

[15] Aldhyani T H H, Al-Nefaie A H, Koundal D. Modeling and diagnosis Parkinson disease by using hand drawing: deep learning model[J]. AIMS Mathematics, 2024, 9(3): 6850-6877.

[16] Farhah N. Utilizing deep learning models in an intelligent spiral drawing classification system for Parkinson’s disease classification[J]. Frontiers in Medicine, 2024, 11: 1453743.

[17] Bi X, Liu J, Meng F, et al. Diagnosis of Parkinson’s disease by Archimedes spiral dynamic handwriting analysis based on Dual-head Multimodal Networks[J]. Biomedical Signal Processing and Control, 2026, 112: 108448.

[18] Sahin E, Akman Aydin E. A Multimodal Convolutional Neural Network Model for Parkinson’s Disease Diagnosis Based on Fused Handwriting Dynamics Signals[J]. Journal of Medical Systems, 2025, 49(1): 132.

[19] Arevalo J, Solorio T, Montes-y-Gómez M, et al. Gated multimodal units for information fusion[J]. arXiv preprint arXiv:1702.01992, 2017.

[20] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.

[21] Zhao A, Liu Y, Yu X, et al. Artificial intelligence-enabled detection and assessment of Parkinson’s disease using multimodal data: A survey[J]. Information Fusion, 2025, 121: 103175.

[22] Liu Z, Mao H, Wu C Y, et al. A convnet for the 2020s[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 11976-11986.