融合面部特征的机动车驾驶人疲劳检测

冯晓锋; 方斌

doi:10.13433/j.cnki.1003-8728.20200282

融合面部特征的机动车驾驶人疲劳检测

doi: 10.13433/j.cnki.1003-8728.20200282

冯晓锋,
方斌

湖南警察学院交通管理系，长沙　410138

基金项目: 湖南省教育厅重点项目（20A173）与湖南省社会科学成果评审委员会项目（XSP19YBC053）

详细信息

作者简介:
冯晓锋（1980−），教授，博士，研究方向为机器视觉和道路交通安全，brucefxf@163.com

中图分类号: U471.3
计量
- 文章访问数: 125
- HTML全文浏览量: 67
- PDF下载量: 8
- 被引次数: 0
出版历程
- 收稿日期: 2020-06-15
- 刊出日期: 2021-11-05

Online Fatigue Detection of Vehicle Drivers based on Facial Features

Department of Traffic Management, Hunan Police Academy, Changsha 410138, China

摘要

摘要: 为降低因驾驶人疲劳驾驶导致的交通事故，需要开展驾驶人疲劳检测研究。为满足在线实时检测的要求，本文提出了融合面部特征的机动车驾驶人疲劳检测方法，首先通过背景差分缩小检测区域、减少图像金字塔层数等方法对MTCNN人脸检测网络进行优化加速，加速后的速度与之前相比提升了258%。其次通过多级级联的残差回归树对人脸进行特征点检测，得到了人脸的特征点，最后通过融合面部嘴、眼开合度特征的方式建立驾驶人疲劳检测模型并进行训练。实验表明，该检测方法的准确率可达95.4%，每帧检测平均速度64 ms，检测速度快，能满足实时性的要求。
- 背景差分 /
- 疲劳检测 /
- 面部特征 /
- 特征点检测
Abstract: In order to reduce the traffic accidents caused by driver fatigue driving, it is necessary to carry out driver fatigue detection research. In order to meet the requirements of online real-time detection, a fatigue detection method for motor vehicle drivers based on facial features is proposed in this paper. Firstly, the MTCNN (multi-task cascaded convolutional networks) face detection network is optimized and accelerated by reducing the detection area through background difference, reducing the number of pyramid layers of image, etc., and the speed after acceleration is 258% times faster than before. Secondly, the multi-level cascaded residual regression tree is used to detect the feature points of driver' s face, and 68 feature points of the face are obtained. Finally, the driver fatigue detection model is established and trained by combining the features of face mouth and eye opening. The experimental results show that the accuracy of the proposed fatigue detection method can reach 95.4%, the average detection speed of each frame is 64ms, and the detection speed is fast enough for meeting the requirements of real-time
- background difference /
- fatigue test /
- facial features /
- feature point detection

HTML全文

图 1 MTCNN网络人脸检测流程

下载: 全尺寸图片幻灯片

图 2 MTCNN网络加速检测流程

下载: 全尺寸图片幻灯片

图 3 背景差分前后的图像

下载: 全尺寸图片幻灯片

图 4 检测到的68个特征点及编号

下载: 全尺寸图片幻灯片

图 5 眼睛睁开及闭合时的开合度

下载: 全尺寸图片幻灯片

图 6 眼睛开合度随时间变化

下载: 全尺寸图片幻灯片

图 7 嘴巴张开及闭合时的张口度

下载: 全尺寸图片幻灯片

图 8 嘴巴开合度随时间变化

下载: 全尺寸图片幻灯片

图 9 融合面部眼嘴状态的疲劳判定流程

下载: 全尺寸图片幻灯片

图 10 疲劳检测示例

下载: 全尺寸图片幻灯片

表 1 面部各特征对应的特征点序号

特征点	人脸轮廓	左眼	右眼	鼻子	嘴巴
序号	1 ~ 23	37 ~ 42	43 ~ 48	28 ~ 36	49 ~ 68

下载: 导出CSV

表 2 人脸检测加速前后的耗时对比

类别	原图	背景差分后	金字塔层数减少后
平均耗时/s	0.168	0.0698	0.0469
速度提升/%	−	141	258

下载: 导出CSV

表 3 疲劳检测全过程耗时对比

序号	疲劳检测环节	平均耗时/s
1	图像读取及处理	0.0049
2	人脸检测	0.0469
3	特征点定位	0.0027
4	疲劳状态判定	0.0095
5	总时长	0.0640

下载: 导出CSV

表 4 疲劳状态判定准确率分析

序号	眼睛疲劳		嘴部疲劳
序号	行为数	行为检出数	行为数	行为检出数
1	5	5	5	5
2	10	10	7	7
3	7	6	2	2
4	15	13	3	3
5	8	8	3	3
准确率	93.3%	93.3%	100%	100%

下载: 导出CSV

参考文献(24)

[1]	张丽霞, 刘涛, 潘福全, 等. 驾驶员因素对道路交通事故指标的影响分析[J]. 中国安全科学学报, 2014, 24(5): 79-84 ZHANG L X, LIU T, PAN F Q, et al. Analysis of effects of driver factors on road traffic accident indexes[J]. China Safety Science Journal, 2014, 24(5): 79-84 (in Chinese)
[2]	TAKAHASHI I, YOKOYAMA K. Development of a feedback stimulation for drowsy driver using heartbeat rhythms[C]//2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Boston: IEEE. 2011: 4153-4158.
[3]	CHUANG C H, HUANG C S, KO L W, et al. An EEG-based perceptual function integration network for application to drowsy driving[J]. Knowledge-Based Systems, 2015, 80: 143-152
[4]	ZHENG W L, GAO K P, LI G, et al. Vigilance estimation using a wearable EOG device in real driving environment[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(1): 170-184 doi: 10.1109/TITS.2018.2889962
[5]	关伟, 杨柳, 江世雄, 等. 脑电在交通驾驶行为中的应用研究综述[J]. 交通运输系统工程与信息, 2016, 16(3): 35-44 doi: 10.3969/j.issn.1009-6744.2016.03.006 GUAN W, YANG L, JIANG S X, et al. Review on the application of EEG in traffic driving behavior study[J]. Journal of Transportation Systems Engineering and Information Technology, 2016, 16(3): 35-44 (in Chinese) doi: 10.3969/j.issn.1009-6744.2016.03.006
[6]	陈志勇, 杨佩, 彭力, 等. 基于BP神经网络的驾驶员疲劳监测研究[J]. 计算机科学, 2015, 42(S1): 67-69, 93 CHEN Z Y, YANG P, PENG L, et al. Fatigue driving monitoring based on BP neural network[J]. Computer Science, 2015, 42(S1): 67-69, 93 (in Chinese)
[7]	李建平, 牛燕雄, 杨露, 等. 基于人眼状态信息的非接触式疲劳驾驶监测与预警系统[J]. 激光与光电子学进展, 2015, 52(4): 041101 LI J P, NIU Y X, YANG L, et al. Contactless driver fatigue detection and warning system based on eye state information[J]. Laser & Optoelectronics Progress, 2015, 52(4): 041101 (in Chinese)
[8]	耿磊, 袁菲, 肖志涛, 等. 基于面部行为分析的驾驶员疲劳检测方法[J]. 计算机工程, 2018, 44(1): 274-279 doi: 10.3969/j.issn.1000-3428.2018.01.046 GENG L, YUAN F, XIAO Z T, et al. Driver fatigue detection method based on facial behavior analysis[J]. Computer Engineering, 2018, 44(1): 274-279 (in Chinese) doi: 10.3969/j.issn.1000-3428.2018.01.046
[9]	顾王欢, 朱煜, 陈旭东, 等. 基于多尺度池化卷积神经网络的疲劳检测方法研究[J]. 计算机应用研究, 2019, 36(11): 3471-3475 GU W H, ZHU Y, CHEN X D, et al. Driver' s fatigue detection system based on multi-scale pooling convolutional neural networks[J]. Application Research of Computers, 2019, 36(11): 3471-3475 (in Chinese)
[10]	史瑞鹏, 钱屹, 蒋丹妮. 一种基于卷积神经网络的疲劳驾驶检测方法[J]. 计算机应用研究, 2020, 37(11): 3481-3486 SHI R P, QIAN Y, JIANG D N. Fatigue driving detection method based on CNN[J]. Application Research of Computers, 2020, 37(11): 3481-3486 (in Chinese)
[11]	戴诗琪, 曾智勇. 基于深度学习的疲劳驾驶检测算法[J]. 计算机系统应用, 2018, 27(7): 113-120 DAI S Q, ZENG Z Y. Fatigue driving detection algorithm based on deep learning[J]. Computer Systems & Applications, 2018, 27(7): 113-120 (in Chinese)
[12]	ZHANG F, SU J J, GENG L, et al. Driver fatigue detection based on eye state recognition[C]//2017 International Conference on Machine Vision and Information Technology (CMVIT). Singapore: IEEE, 2017: 105-110.
[13]	ZHANG K P, ZHANG Z P, LI Z F, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503 doi: 10.1109/LSP.2016.2603342
[14]	王爱丽, 董宝田, 王泽胜. 融合背景差分的二次重构和内外标记分水岭的行人检测方法[J]. 交通运输系统工程与信息, 2014, 14(4): 66-72, 112 doi: 10.3969/j.issn.1009-6744.2014.04.009 WANG A L, DONG B T, WANG Z S. Pedestrian detection of integrating BS based on quadratic reconstruction and IE marker watershed[J]. Journal of Transportation Systems Engineering and Information Technology, 2014, 14(4): 66-72, 112 (in Chinese) doi: 10.3969/j.issn.1009-6744.2014.04.009
[15]	李兴鑫, 朱力强, 余祖俊. 自适应铁路场景前景目标检测[J]. 交通运输系统工程与信息, 2020, 20(2): 83-90 LI X X, ZHU L Q, YU Z J. Adaptive foreground object detection in railway scene[J]. Journal of Transportation Systems Engineering and Information Technology, 2020, 20(2): 83-90 (in Chinese)
[16]	KAZEMI V, SULLIVAN J. One millisecond face alignment with an ensemble of regression trees[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 1867-1874.
[17]	LAL S K L, CRAIG A. A critical review of the psychophysiology of driver fatigue[J]. Biological Psychology, 2001, 55(3): 173-194 doi: 10.1016/S0301-0511(00)00085-5
[18]	袁翔, 孙香梅. 疲劳驾驶检测方法研究进展[J]. 汽车工程学报, 2012, 2(3): 157-164 doi: 10.3969/j.issn.2095-1469.2012.03.001 YUAN X, SUN X M. Development of driver fatigue detection method research[J]. Chinese Journal of Automotive Engineering, 2012, 2(3): 157-164 (in Chinese) doi: 10.3969/j.issn.2095-1469.2012.03.001
[19]	SOUKUPOVÁ T, CECH J. Real-time eye blink detection using facial landmarks[C]//21st Computer Vision Winter Workshop. Rimske Toplice: CVWW, 2016: 1-8.
[20]	REDDY B, KIM Y H, YUN S, et al. Real-time driver drowsiness detection for embedded system using model compression of deep neural networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu: IEEE, 2017: 438-445.
[21]	刘炜煌, 钱锦浩, 姚增伟, 等. 基于多面部特征融合的驾驶员疲劳检测算法[J]. 计算机系统应用, 2018, 27(10): 177-182 LIU W H, QIAN J H, YAO Z W, et al. Driver fatigue detection algorithm based on multi-facial feature fusion[J]. Computer Systems & Applications, 2018, 27(10): 177-182 (in Chinese)
[22]	Office of Motor Carrier Research and Standards. PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance[R]. Washington: US Department of Transportation, Federal Highway Administration, 1998.
[23]	TRUTSCHEL U, SIROIS B, SOMMER D, et al. Perclos: an alertness measure of the past[C]//2011 Driving Assessment Conference. Lake Tahoe: Public Policy Center, University of Iowa, 2011: 172-179.
[24]	王霞, 仝美娇, 王蒙军. 基于嘴部内轮廓特征的疲劳检测[J]. 科学技术与工程, 2016, 16(26): 240-244 doi: 10.3969/j.issn.1671-1815.2016.26.039 WANG X, TONG M J, WANG M J. Fatigue detection based on the inner profile characteristics of the mouth[J]. Science Technology and Engineering, 2016, 16(26): 240-244 (in Chinese) doi: 10.3969/j.issn.1671-1815.2016.26.039