Optimizing Bi-LSTM networks for improved lung cancer detection accuracy

Abstract

Lung cancer remains a leading cause of cancer-related deaths worldwide, with low survival rates often attributed to late-stage diagnosis. To address this critical health challenge, researchers have developed computer-aided diagnosis (CAD) systems that rely on feature extraction from medical images. However, accurately identifying the most informative image features for lung cancer detection remains a significant challenge. This study aimed to compare the effectiveness of both hand-crafted and deep learning-based approaches for lung cancer diagnosis. We employed traditional hand-crafted features, such as Gray Level Co-occurrence Matrix (GLCM) features, in conjunction with traditional machine learning algorithms. To explore the potential of deep learning, we also optimized and implemented a Bidirectional Long Short-Term Memory (Bi-LSTM) network for lung cancer detection. The results revealed that the highest performance using hand-crafted features was achieved by extracting GLCM features and utilizing Support Vector Machine (SVM) with different kernels, reaching an accuracy of 99.78% and an AUC of 0.999. However, the deep learning Bi-LSTM network surpassed both methods, achieving an accuracy of 99.89% and an AUC of 1.0000. These findings suggest that the proposed methodology, combining hand-crafted features and deep learning, holds significant promise for enhancing early lung cancer detection and ultimately improving diagnosis systems.