Integrating DRN-RF with computer vision for detection of control room operator’s mental fatigue

Abstract

Control room operators encounter a substantial risk of mental fatigue, which can reduce their human reliability by diminishing concentration and responsiveness, leading to unsafe operations. There is value in detection of individuals’ mental fatigue status in the workplace. This study introduces a new method for mental fatigue detection (MFD) that combines computer vision and machine learning. Traditional methods for MFD typically rely on multi-dimensional data for fatigue analysis and detection, which can be challenging to apply in a real situation. The traditional methods such as the use of biological data, e.g., electrocardiograms, require operators to be in constant contact with sensors, while this study utilizes computer vision to collect facial data, and a machine learning model to assess fatigue states. The developed machine learning method consists both Deep Residual Network and Random Forest (DRN-RF). A comparison with existing MFD methods, including K Nearest Neighbors and Gradient Boosting Machine, has been carried out. The results show that the accuracy of the DRN-RF model reaches 94.2% and the deviation is 0.004. Evidently, the DRN-RF model demonstrates high accuracy and stability. Overall, the proposed method has the potential to contribute to improving the safety of process system operations, particularly in the aspect of human factor management.