留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

融合面部特征的机动车驾驶人疲劳检测

冯晓锋 方斌

冯晓锋,方斌. 融合面部特征的机动车驾驶人疲劳检测[J]. 机械科学与技术,2021,40(11):1767-1772 doi: 10.13433/j.cnki.1003-8728.20200282
引用本文: 冯晓锋,方斌. 融合面部特征的机动车驾驶人疲劳检测[J]. 机械科学与技术,2021,40(11):1767-1772 doi: 10.13433/j.cnki.1003-8728.20200282
FENG Xiaofeng, FANG Bin. Online Fatigue Detection of Vehicle Drivers based on Facial Features[J]. Mechanical Science and Technology for Aerospace Engineering, 2021, 40(11): 1767-1772. doi: 10.13433/j.cnki.1003-8728.20200282
Citation: FENG Xiaofeng, FANG Bin. Online Fatigue Detection of Vehicle Drivers based on Facial Features[J]. Mechanical Science and Technology for Aerospace Engineering, 2021, 40(11): 1767-1772. doi: 10.13433/j.cnki.1003-8728.20200282

融合面部特征的机动车驾驶人疲劳检测

doi: 10.13433/j.cnki.1003-8728.20200282
基金项目: 湖南省教育厅重点项目(20A173)与湖南省社会科学成果评审委员会项目(XSP19YBC053)
详细信息
    作者简介:

    冯晓锋(1980−),教授,博士,研究方向为机器视觉和道路交通安全,brucefxf@163.com

  • 中图分类号: U471.3

Online Fatigue Detection of Vehicle Drivers based on Facial Features

  • 摘要: 为降低因驾驶人疲劳驾驶导致的交通事故,需要开展驾驶人疲劳检测研究。为满足在线实时检测的要求,本文提出了融合面部特征的机动车驾驶人疲劳检测方法,首先通过背景差分缩小检测区域、减少图像金字塔层数等方法对MTCNN人脸检测网络进行优化加速,加速后的速度与之前相比提升了258%。其次通过多级级联的残差回归树对人脸进行特征点检测,得到了人脸的特征点,最后通过融合面部嘴、眼开合度特征的方式建立驾驶人疲劳检测模型并进行训练。实验表明,该检测方法的准确率可达95.4%,每帧检测平均速度64 ms,检测速度快,能满足实时性的要求。
  • 图  1  MTCNN网络人脸检测流程

    图  2  MTCNN网络加速检测流程

    图  3  背景差分前后的图像

    图  4  检测到的68个特征点及编号

    图  5  眼睛睁开及闭合时的开合度

    图  6  眼睛开合度随时间变化

    图  7  嘴巴张开及闭合时的张口度

    图  8  嘴巴开合度随时间变化

    图  9  融合面部眼嘴状态的疲劳判定流程

    图  10  疲劳检测示例

    表  1  面部各特征对应的特征点序号

    特征点人脸
    轮廓
    左眼右眼鼻子嘴巴
    序号1 ~ 2337 ~ 4243 ~ 4828 ~ 3649 ~ 68
    下载: 导出CSV

    表  2  人脸检测加速前后的耗时对比

    类别原图背景
    差分后
    金字塔层数减少后
    平均耗时/s0.1680.06980.0469
    速度提升/%141258
    下载: 导出CSV

    表  3  疲劳检测全过程耗时对比

    序号疲劳检测环节平均耗时/s
    1图像读取及处理0.0049
    2人脸检测0.0469
    3特征点定位0.0027
    4疲劳状态判定0.0095
    5总时长0.0640
    下载: 导出CSV

    表  4  疲劳状态判定准确率分析

    序号眼睛疲劳嘴部疲劳
    行为数行为检出数行为数行为检出数
    1 5 5 5 5
    2 10 10 7 7
    3 7 6 2 2
    4 15 13 3 3
    5 8 8 3 3
    准确率 93.3% 93.3% 100% 100%
    下载: 导出CSV
  • [1] 张丽霞, 刘涛, 潘福全, 等. 驾驶员因素对道路交通事故指标的影响分析[J]. 中国安全科学学报, 2014, 24(5): 79-84

    ZHANG L X, LIU T, PAN F Q, et al. Analysis of effects of driver factors on road traffic accident indexes[J]. China Safety Science Journal, 2014, 24(5): 79-84 (in Chinese)
    [2] TAKAHASHI I, YOKOYAMA K. Development of a feedback stimulation for drowsy driver using heartbeat rhythms[C]//2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Boston: IEEE. 2011: 4153-4158.
    [3] CHUANG C H, HUANG C S, KO L W, et al. An EEG-based perceptual function integration network for application to drowsy driving[J]. Knowledge-Based Systems, 2015, 80: 143-152
    [4] ZHENG W L, GAO K P, LI G, et al. Vigilance estimation using a wearable EOG device in real driving environment[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(1): 170-184 doi: 10.1109/TITS.2018.2889962
    [5] 关伟, 杨柳, 江世雄, 等. 脑电在交通驾驶行为中的应用研究综述[J]. 交通运输系统工程与信息, 2016, 16(3): 35-44 doi: 10.3969/j.issn.1009-6744.2016.03.006

    GUAN W, YANG L, JIANG S X, et al. Review on the application of EEG in traffic driving behavior study[J]. Journal of Transportation Systems Engineering and Information Technology, 2016, 16(3): 35-44 (in Chinese) doi: 10.3969/j.issn.1009-6744.2016.03.006
    [6] 陈志勇, 杨佩, 彭力, 等. 基于BP神经网络的驾驶员疲劳监测研究[J]. 计算机科学, 2015, 42(S1): 67-69, 93

    CHEN Z Y, YANG P, PENG L, et al. Fatigue driving monitoring based on BP neural network[J]. Computer Science, 2015, 42(S1): 67-69, 93 (in Chinese)
    [7] 李建平, 牛燕雄, 杨露, 等. 基于人眼状态信息的非接触式疲劳驾驶监测与预警系统[J]. 激光与光电子学进展, 2015, 52(4): 041101

    LI J P, NIU Y X, YANG L, et al. Contactless driver fatigue detection and warning system based on eye state information[J]. Laser & Optoelectronics Progress, 2015, 52(4): 041101 (in Chinese)
    [8] 耿磊, 袁菲, 肖志涛, 等. 基于面部行为分析的驾驶员疲劳检测方法[J]. 计算机工程, 2018, 44(1): 274-279 doi: 10.3969/j.issn.1000-3428.2018.01.046

    GENG L, YUAN F, XIAO Z T, et al. Driver fatigue detection method based on facial behavior analysis[J]. Computer Engineering, 2018, 44(1): 274-279 (in Chinese) doi: 10.3969/j.issn.1000-3428.2018.01.046
    [9] 顾王欢, 朱煜, 陈旭东, 等. 基于多尺度池化卷积神经网络的疲劳检测方法研究[J]. 计算机应用研究, 2019, 36(11): 3471-3475

    GU W H, ZHU Y, CHEN X D, et al. Driver' s fatigue detection system based on multi-scale pooling convolutional neural networks[J]. Application Research of Computers, 2019, 36(11): 3471-3475 (in Chinese)
    [10] 史瑞鹏, 钱屹, 蒋丹妮. 一种基于卷积神经网络的疲劳驾驶检测方法[J]. 计算机应用研究, 2020, 37(11): 3481-3486

    SHI R P, QIAN Y, JIANG D N. Fatigue driving detection method based on CNN[J]. Application Research of Computers, 2020, 37(11): 3481-3486 (in Chinese)
    [11] 戴诗琪, 曾智勇. 基于深度学习的疲劳驾驶检测算法[J]. 计算机系统应用, 2018, 27(7): 113-120

    DAI S Q, ZENG Z Y. Fatigue driving detection algorithm based on deep learning[J]. Computer Systems & Applications, 2018, 27(7): 113-120 (in Chinese)
    [12] ZHANG F, SU J J, GENG L, et al. Driver fatigue detection based on eye state recognition[C]//2017 International Conference on Machine Vision and Information Technology (CMVIT). Singapore: IEEE, 2017: 105-110.
    [13] ZHANG K P, ZHANG Z P, LI Z F, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503 doi: 10.1109/LSP.2016.2603342
    [14] 王爱丽, 董宝田, 王泽胜. 融合背景差分的二次重构和内外标记分水岭的行人检测方法[J]. 交通运输系统工程与信息, 2014, 14(4): 66-72, 112 doi: 10.3969/j.issn.1009-6744.2014.04.009

    WANG A L, DONG B T, WANG Z S. Pedestrian detection of integrating BS based on quadratic reconstruction and IE marker watershed[J]. Journal of Transportation Systems Engineering and Information Technology, 2014, 14(4): 66-72, 112 (in Chinese) doi: 10.3969/j.issn.1009-6744.2014.04.009
    [15] 李兴鑫, 朱力强, 余祖俊. 自适应铁路场景前景目标检测[J]. 交通运输系统工程与信息, 2020, 20(2): 83-90

    LI X X, ZHU L Q, YU Z J. Adaptive foreground object detection in railway scene[J]. Journal of Transportation Systems Engineering and Information Technology, 2020, 20(2): 83-90 (in Chinese)
    [16] KAZEMI V, SULLIVAN J. One millisecond face alignment with an ensemble of regression trees[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 1867-1874.
    [17] LAL S K L, CRAIG A. A critical review of the psychophysiology of driver fatigue[J]. Biological Psychology, 2001, 55(3): 173-194 doi: 10.1016/S0301-0511(00)00085-5
    [18] 袁翔, 孙香梅. 疲劳驾驶检测方法研究进展[J]. 汽车工程学报, 2012, 2(3): 157-164 doi: 10.3969/j.issn.2095-1469.2012.03.001

    YUAN X, SUN X M. Development of driver fatigue detection method research[J]. Chinese Journal of Automotive Engineering, 2012, 2(3): 157-164 (in Chinese) doi: 10.3969/j.issn.2095-1469.2012.03.001
    [19] SOUKUPOVÁ T, CECH J. Real-time eye blink detection using facial landmarks[C]//21st Computer Vision Winter Workshop. Rimske Toplice: CVWW, 2016: 1-8.
    [20] REDDY B, KIM Y H, YUN S, et al. Real-time driver drowsiness detection for embedded system using model compression of deep neural networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu: IEEE, 2017: 438-445.
    [21] 刘炜煌, 钱锦浩, 姚增伟, 等. 基于多面部特征融合的驾驶员疲劳检测算法[J]. 计算机系统应用, 2018, 27(10): 177-182

    LIU W H, QIAN J H, YAO Z W, et al. Driver fatigue detection algorithm based on multi-facial feature fusion[J]. Computer Systems & Applications, 2018, 27(10): 177-182 (in Chinese)
    [22] Office of Motor Carrier Research and Standards. PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance[R]. Washington: US Department of Transportation, Federal Highway Administration, 1998.
    [23] TRUTSCHEL U, SIROIS B, SOMMER D, et al. Perclos: an alertness measure of the past[C]//2011 Driving Assessment Conference. Lake Tahoe: Public Policy Center, University of Iowa, 2011: 172-179.
    [24] 王霞, 仝美娇, 王蒙军. 基于嘴部内轮廓特征的疲劳检测[J]. 科学技术与工程, 2016, 16(26): 240-244 doi: 10.3969/j.issn.1671-1815.2016.26.039

    WANG X, TONG M J, WANG M J. Fatigue detection based on the inner profile characteristics of the mouth[J]. Science Technology and Engineering, 2016, 16(26): 240-244 (in Chinese) doi: 10.3969/j.issn.1671-1815.2016.26.039
  • 加载中
图(10) / 表(4)
计量
  • 文章访问数:  187
  • HTML全文浏览量:  95
  • PDF下载量:  9
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-06-15
  • 刊出日期:  2021-11-05

目录

    /

    返回文章
    返回