Entity Recognition in Stroke Medical Records using Sparse Attention Mechanisms

Liu Xinyu; Yao Quanzhu; Zhou Ze

Current Chinese named-entity recognition (NER) models for stroke medical records face challenges in processing long texts, including excessive memory consumption and information loss. To address these issues, this study proposes a domain-specific NER model for stroke medical records using sparse attention mechanisms. First, a sparse attention mechanism—comprising sliding window attention with sparse global attention—is integrated into the RoBERTa embedding layer to enhance long-text processing efficiency while preserving critical information. Subsequently, a bidirectional long short-term memory (BiLSTM) network captures contextual dependencies for feature extraction. Finally, a conditional random field (CRF) layer models label transition probabilities based on positional features, generating globally optimal label sequences. Comparative experiments on two distinct datasets demonstrate that the proposed model achieves F1-score improvements of 1.24% and 2.30%, respectively, over the baseline RoBERTa-BiLSTM-CRF architecture. Additional experiments on long-text entity recognition reveal a 1.97% F1-score gain compared to the RoBERTa model without sparse attention, confirming the mechanism’s efficacy in improving long-text processing and recognition accuracy.

Select Volume / Issues:

• Volume 16, Issue 2, Mar. 2025

Year:

2025

Type of Publication:

Article

Keywords:

Named Entity Recognition; Stroke Disease; Sparse Attention Mechanism; Attention Mechanism

Journal:

IJECCE

Volume:

16

Number:

2

Pages:

17-27

Month:

March

ISSN:

2249-071X