视觉手势识别的移动机器人手势控制系统设计

龙樟; 卢成娅; 李国鹏; 张维烈; 温飞娟; 李波

doi:10.13433/j.cnki.1003-8728.20200449

视觉手势识别的移动机器人手势控制系统设计

doi: 10.13433/j.cnki.1003-8728.20200449

1.
西南石油大学工程学院, 四川南充 637000
2.
西南石油大学信息学院, 四川南充 637000

基金项目:

南充市市校科技战略合作项目 18SXHZ0008

南充市市校科技战略合作项目 19SXHZ0036

南充市市校科技战略合作项目 19SXHZ0037

详细信息

作者简介:
龙樟(1992-), 讲师, 硕士研究生, 研究方向为机器视觉、机器人技术, longzhang@swpu.edu.cn

通讯作者:
温飞娟, 讲师, 硕士研究生, wenfeijuan@swpu.edu.cn

中图分类号: TP391.4
计量
- 文章访问数: 135
- HTML全文浏览量: 353
- PDF下载量: 42
- 被引次数: 0
出版历程
- 收稿日期: 2020-10-23
- 刊出日期: 2022-08-25

Design on Mobile Robot Gesture Control System with Visual Gesture Recognition

1.
School of Engineering, Southwest Petroleum University, Nanchong 637000, Sichuan, China
2.
School of Information, Southwest Petroleum University, Nanchong 637000, Sichuan, China

摘要

摘要: 为了实现机器人控制人机交互的智能化与多功能化，提出了基于视觉手势识别的移动机器人手势控制系统。采用麦克纳姆轮搭建了全向四轮移动机器人平台，以主从控制作为机器人整体控制构架，采用PC机搭建了上位机视觉手势识别系统，采用树莓派设计了下位机机器人运动控制系统，并利用较高可靠性的TCP协议实现了上、下位机WIFI无线通信。采用改进的VGG16网络模型设计了手势识别算法，利用建立的手势图像数据库对网络模型进行训练和测试，通过对改进结构VGG16网络的应用，实现了高准确度的手势图像识别。通过实验验证了视觉识别手势控制移动机器人方案的可行性，为手势控制的应用提供了理论参考。
- 手势识别 /
- 深度学习 /
- 卷积神经网络 /
- 手势控制
Abstract: In order to realize the intellectualization and multi-functionalization of robot control human-computer interaction, a mobile robot gesture control system based on visual gesture recognition is proposed in this paper. An omnidirectional four-wheel mobile robot was put up with Mecanum wheels. The master-slave control was used as the robot control architecture. The visual gesture recognition system as host computer was built with PC. The robot motion control system of the lower computer was designed with Raspberry Pi. And using higher reliability TCP protocol achieves WIFI wireless communication between the host and lower computer. Using the improved VGG16 network model, a gesture recognition algorithm was designed, and we used the established gesture image database to train and test the network model. Through the application of the improved VGG16 network, high-precision gesture image recognition was realized. The feasibility of this scheme that uses the visual recognition gesture to control mobile robot was verified through experiments, which provides a theoretical reference for the application of gesture control.
- gesture recognition /
- deep learning /
- convolutional neural network(CNN) /
- gesture control

HTML全文

图 1 移动机器人平台

下载: 全尺寸图片幻灯片

图 2 系统整体架构

下载: 全尺寸图片幻灯片

图 3 手势图像预处理

下载: 全尺寸图片幻灯片

图 4 手势数据集制作

下载: 全尺寸图片幻灯片

图 5 手势库的建立和分类

下载: 全尺寸图片幻灯片

图 6 网络模型训练与调优流程

下载: 全尺寸图片幻灯片

图 7 改进VGG16网络结构

下载: 全尺寸图片幻灯片

图 8 手势识别效果图

下载: 全尺寸图片幻灯片

图 9 手势位姿造成的识别困难示例

下载: 全尺寸图片幻灯片

图 10 手势控制移动机器人测试实验

下载: 全尺寸图片幻灯片

表 1 手势指令与运动方式对应关系

参考文献(19)

[1]	QIAN K, NIU J, YANG H. Developing a gesture based remote human-robot interaction system using kinect[J]. International Journal of Smart Home, 2013, 7(4): 203-208
[2]	熊友军, 李世其, 王文涛. 基于数据手套驱动的虚拟机器人操作技术[J]. 机械科学与技术, 2004, 23(12): 1433-1436 doi: 10.3321/j.issn:1003-8728.2004.12.013 XIONG Y J, LI S Q, WANG W T. Operating technology of virtual robot based on data glove drive[J]. Mechanical Science and Technology for Aerospace Engineering, 2004, 23(12): 1433-1436 (in Chinese) doi: 10.3321/j.issn:1003-8728.2004.12.013
[3]	TANG D H, CHANG H J, TEJANI A, et al. Latent regression forest: structured estimation of 3D hand poses[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(7): 1374-1387 doi: 10.1109/TPAMI.2016.2599170
[4]	WANG R Y, POPOVIĆ J. Real-time hand-tracking with a color glove[J]. ACM Transactions on Graphics, 2009, 28(3): 63
[5]	CAMBUIM L F S, MACIEIR R M, NETO F M P, et al. An efficient static gesture recognizer embedded system based on ELM pattern recognition algorithm[J]. Journal of Systems Architecture, 2016, 68: 1-16 doi: 10.1016/j.sysarc.2016.06.002
[6]	田秋红, 杨慧敏, 梁庆龙, 等. 视觉动态手势识别综述[J]. 浙江理工大学学报, 2020, 43(4): 557-569 https://www.cnki.com.cn/Article/CJFDTOTAL-ZJSG202004018.htm TIAN Q H, YANG H M, LIANG Q L, et al. Overview on vision-based dynamic gesture recognition[J]. Journal of Zhejiang Sci-Tech University, 2020, 43(4): 557-569 (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZJSG202004018.htm
[7]	NASEER T, STURM J, CREMERS D. Followme: person following and gesture recognition with a quadrocopter[C]// Proceedings of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo: IEEE, 2013: 624-630
[8]	覃文军, 杨金柱, 王力, 等. 基于Kalman滤波器与肤色模型的手势跟踪方法[J]. 东北大学学报(自然科学版), 2013, 34(4): 474-477 doi: 10.3969/j.issn.1005-3026.2013.04.005 TAN W J, YANG J Z, WANG L, et al. Hand gesture tracking method based on Kalman filter and skin color feature[J]. Journal of Northeastern University (Natural Science), 2013, 34(4): 474-477 (in Chinese) doi: 10.3969/j.issn.1005-3026.2013.04.005
[9]	KONE C ˇ N Y ' J, HAGARA M. One-shot-learning gesture recognition using HOG-HOF features[J]. Journal of Machine Learning Research, 2014, 15: 2513-2532
[10]	HAZMOUNE S, BOUGAMOUZA F, MAZOUZI S, et al. A new hybrid framework based on Hidden Markov models and K-nearest neighbors for speech recognition[J]. International Journal of Speech Technology, 2018, 21(3): 689-704 doi: 10.1007/s10772-018-9535-4
[11]	MANZI A, CAVALLO F, DARIO P. A 3D human posture approach for activity recognition based on depth camera[C]// European Conference on Computer Vision. Amsterdam: Springer, 2016: 432-447
[12]	Tsironi E, Barros P, Weber C, et al. An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition[J]. Neurocomputing, 2017, 268: 76-86 doi: 10.1016/j.neucom.2016.12.088
[13]	张博言, 钟勇, 李振东. 基于动态模式和卷积特征的单目标跟踪算法[J]. 西北工业大学学报, 2019, 37(6): 1310-1319 https://www.cnki.com.cn/Article/CJFDTOTAL-XBGD201906027.htm ZHANG B Y, ZHONG Y, LI Z D. A visual object tracking algorithm based on dynamics pattern and convolutional feature[J]. Journal of Northwestern Polytechnical University, 2019, 37(6): 1310-1319 (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XBGD201906027.htm
[14]	马乐乐, 李照洋, 董嘉蓉, 等. 基于计算机视觉及深度学习的无人机手势控制系统[J]. 计算机工程与科学, 2018, 40(5): 872-879 doi: 10.3969/j.issn.1007-130X.2018.05.016 MA L L, LI Z Y, DONG J R, et al. UAV gesture control system based on computer vision and deep learning[J]. Computer Engineering & Science, 2018, 40(5): 872-879 (in Chinese) doi: 10.3969/j.issn.1007-130X.2018.05.016
[15]	薛文奎. 基于手势识别的采摘机器人智能控制系统[J]. 农机化研究, 2020, 42(7): 249-253 https://www.cnki.com.cn/Article/CJFDTOTAL-NJYJ202007044.htm XUE W K. Intelligent control system of picking robot based on visual gesture recognition[J]. Journal of Agricultural Mechanization Research, 2020, 42(7): 249-253 (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-NJYJ202007044.htm
[16]	衣世东. 基于深度学习的图像识别算法研究[D]. 郑州: 战略支援部队信息工程大学, 2018 YI S D. Image recognition based on deep learning algorithm[D]. Zhengzhou: Information Engineering University, 2018 (in Chinese)
[17]	WANG N, WANG Y Y, ER J M. Review on deep learning techniques for marine object recognition: architectures and algorithms[J]. Control Engineering Practice, 2022, 118: 104458
[18]	QASSIM H, VERMA A, FEINZIMER D. Compressed residual-VGG16 CNN model for big data places image recognition[C]//IEEE 8th Annual Computing and Communication Workshop and Conference. Las Vegas: IEEE, 2018: 169-175
[19]	王海云, 王剑平, 罗付华. 融合多层次特征Faster R-CNN的金属板带材表面缺陷检测研究[J]. 机械科学与技术, 2021, 40(2): 262-269 doi: 10.13433/j.cnki.1003-8728.20200024 WANG H Y, WANG J P, LUO F H. Study on surface defect detection of metal sheet and strip using faster R-CNN with multilevel feature[J]. Mechanical Science and Technology for Aerospace Engineering, 2021, 40(2): 262-269 (in Chinese) doi: 10.13433/j.cnki.1003-8728.20200024