增强现实环境下装配操作感知方法研究

梁剑斌; 何汉武; 吴悦明; 胡兆勇; 徐梓麒

doi:10.13433/j.cnki.1003-8728.20220217

增强现实环境下装配操作感知方法研究

doi: 10.13433/j.cnki.1003-8728.20220217

梁剑斌^1,,
何汉武^{1, 2, ,},
吴悦明¹,
胡兆勇¹,
徐梓麒¹

1.
广东工业大学机电工程学院，广州　510006
2.
广东工贸职业技术学院，广州　510006

基金项目: 国家重点研发计划(2018YFB1004902)

详细信息

作者简介:
梁剑斌，博士研究生，tankhero@126.com

通讯作者:
何汉武，教授，博士，hwhe@gdut.edu.cn

中图分类号: TG391.9
计量
- 文章访问数: 21
- HTML全文浏览量: 11
- PDF下载量: 7
- 被引次数: 0
出版历程
- 收稿日期: 2021-12-13
- 网络出版日期: 2024-03-08
- 刊出日期: 2024-02-01

An Assembly Operation Perception Model in Augmented Reality Environment

LIANG Jianbin^1
,,
HE Hanwu^{1, 2
, ,},
WU Yueming¹,
HU Zhaoyong¹,
XU Ziqi¹

1.
School of Electromechanical Engineering, Guangdong University of Technology, Guangzhou 510006, China
2.
Guangdong Polytechnic of Industry and Commerce, Guangzhou 510006, China

摘要

摘要: 现有的自然虚拟装配操作交互模型对双手协同操作和单步操作工作量的表达存在不足，针对这个问题，提出增强现实装配操作感知模型，实现机械装配操作参数化表达。提出基于装配运动特征的虚拟物体位姿计算方法，建立模型的真实手部元素与被操作虚拟物体元素的运动关联，满足模型对虚拟物体位姿求解的需求。提出基于模型元素的模型关键状态计算方法，满足模型运行需求。开发了应用案例，实验表明，模型可表达五种常见装配操作，满足装配培训需求。
- 装配操作感知 /
- 装配运动特征 /
- 虚拟装配 /
- 增强现实
Abstract: The existing virtual assembly system’s natural assembly operation and interaction models have defects for expressing the workload of two-handed cooperative operation and single-step operation. To solve this problem, an assembly operation perception model in its augmented reality environment is proposed. The model can parametrically express the mechanical assembly operation. A virtual object pose calculation method based on assembly motion characteristics is proposed. The method is based on the motion correlation between the elements of the model, namely the real hand and the manipulated virtual object. The calculation methods of the model's key states based on its elements are proposed to meet the requirements of the model's operation. An application case is developed. The experimental results show that the assembly operation perception model can express five common assembly operations and satisfy the requirements for assembly training.
- assembly operation perception /
- assembly motion characteristics /
- virtual assembly /
- augmented reality

HTML全文

图 1 手部元素与真实手部的对应关系

Figure 1. Correspondence between hand elements and a real hand

下载: 全尺寸图片幻灯片

图 2 装配操作感知模型元素空间关系

Figure 2. Spatial relationship among elements in the assembly operation perception model

下载: 全尺寸图片幻灯片

图 3 单步操作在t时刻工作流程

Figure 3. Workflow of single operation at time t

下载: 全尺寸图片幻灯片

图 4 汽车变速箱三维模型

Figure 4. 3D model of automobile gearbox

下载: 全尺寸图片幻灯片

图 5 交互场景实景图

Figure 5. An interactive scene

下载: 全尺寸图片幻灯片

图 6 放置操作过程

Figure 6. Process of set operation

下载: 全尺寸图片幻灯片

图 7 锤击操作过程

Figure 7. Hammering operation process

下载: 全尺寸图片幻灯片

图 8 压入操作过程

Figure 8. Push-in operation process

下载: 全尺寸图片幻灯片

图 9 悬停操作过程

Figure 9. Hover operation process

下载: 全尺寸图片幻灯片

图 10 扭动扳手操作过程

Figure 10. Twisting wrench operation process

下载: 全尺寸图片幻灯片

表 1 几何约束与虚拟物体运动方式的对应关系

Table 1. Correspondence between geometric constraints and virtual object movement modes

运动方式	p_jfc	v_jfc	q_jfc
自由移动	无取值	无取值	无取值
沿轴线移动（可绕轴转）	p_jfc	v_jfc	1
沿轴线移动（不可旋转）	p_jfc	v_jfc	0
绕点旋转	p_jfc	无取值	无取值

下载: 导出CSV

表 2 放置操作参数

Table 2. Parameters of set operation

参数	j=0
VO	vo₀
DM	dm₀={o_0dm-x_0dmy_0dmz_0dm, M_0dm0, M_0dm, null, r_0dm, α_0dm}

下载: 导出CSV

表 3 锤击操作参数

Table 3. Parameters of hammering operation

参数	j=0	j=1
VO	vo₀	vo₁
DM	dm₀= {o_0dm-x_0dmy_0dmz_0dm, M_0dm0, M_0dm, null, r_0dm, α_0dm}	dm₁= {o_1dm-x_1dmy_1dmz_1dm, M_1dm0, M_1dm, h_1dm, r_1dm, α_1dm}
cv	{n_cv, null}

下载: 导出CSV

表 4 压入操作参数

Table 4. Parameters of push-in operation

参数	j=0
VO	vo₀
FC	fc₀={p_0fc, v_0fc, 0}
DM	dm₀={o_0dm-x_0dmy_0dmz_0dm, M_0dm0, M_0dm, h_0dm, r_0dm, α_0dm}

下载: 导出CSV

表 5 悬停操作参数

Table 5. Parameters of hover operation

参数	j=0	j=1
VO	vo₀	vo₁
DM	dm₀= {o_0dm-x_0dmy_0dmz_0dm, M_0dm0, M_0dm, null, r_0dm, α_0dm}	dm₁= {o_1dm-x_1dmy_1dmz_1dm, M_1dm0, M_1dm, null, r_1dm, α_1dm}

下载: 导出CSV

表 6 扭动扳手操作参数

Table 6. Parameters of twisting wrench operation

参数	j=0
VO	vo₀
FC	fc₀={p_0fc, null, null}
DM	dm₀={o_0dm-x_0dmy_0dmz_0dm, M_0dm0, M_0dm, null, r_0dm, null}
cv	{null, α_cv}

下载: 导出CSV

参考文献(15)

[1]	朱正伟, 祝磊, 饶鹏. 面向工业培训应用的虚拟现实装配系统研究[J]. 现代电子技术, 2019, 42(20): 58-63. doi: 10.16652/j.issn.1004-373x.2019.20.015 ZHU Z W, ZHU L, RAO P. Research on virtual reality assembly system for industrial training application[J]. Modern Electronics Technique, 2019, 42(20): 58-63. (in Chinese) doi: 10.16652/j.issn.1004-373x.2019.20.015
[2]	HOEDT S, CLAEYS A, VAN LANDEGHEM H, et al. The evaluation of an elementary virtual training system for manual assembly[J]. International Journal of Production Research, 2017, 55(24): 7496-7508. doi: 10.1080/00207543.2017.1374572
[3]	夏平均, 郎跃东, 程鹏, 等. 面向大型复杂产品的虚拟装配技术研究[J]. 机械科学与技术, 2008, 27(8): 1000-1004. doi: 10.3321/j.issn:1003-8728.2008.08.005 XIA P J, LANG Y D, CHENG P, et al. Study on virtual assembly technology for complex products[J]. Mechanical Science and Technology for Aerospace Engineering, 2008, 27(8): 1000-1004. (in Chinese) doi: 10.3321/j.issn:1003-8728.2008.08.005
[4]	ONG S K, WANG Z B. Augmented assembly technologies based on 3D bare-hand interaction[J]. CIRP Annals, 2011, 60(1): 1-4. doi: 10.1016/j.cirp.2011.03.001
[5]	绪玉花, 李静蓉. 面向虚拟装配的虚拟手交互技术研究[J]. 机械设计与制造, 2014(5): 262-266. doi: 10.3969/j.issn.1001-3997.2014.05.080 XU Y H, LI J R. Research and implementation of virtual hand interaction in virtual mechanical assembly[J]. Machinery Design & Manufacture, 2014(5): 262-266. (in Chinese) doi: 10.3969/j.issn.1001-3997.2014.05.080
[6]	黄培德, 王琳, 萧筝, 等. VR环境下的虚拟手自然交互方法研究[J]. 机械设计与制造, 2021(7): 268-271. doi: 10.3969/j.issn.1001-3997.2021.07.063 HUANG P D, WANG L, XIAO Z, et al. Research on natural interaction method of virtual hand in VR environment[J]. Machinery Design & Manufacture, 2021(7): 268-271. (in Chinese) doi: 10.3969/j.issn.1001-3997.2021.07.063
[7]	胡弘, 晁建刚, 林万洪, 等. 基于虚拟手交互的航天员虚拟装配训练仿真方法[J]. 计算机应用, 2015, 35(S2): 200-203. HU H, CHAO J G, LIN W H, et al. Astronaut virtual assembly training simulation based on virtual hand interaction[J]. Journal of Computer Applications, 2015, 35(S2): 200-203. (in Chinese)
[8]	包文运, 胡建垠, 黄培德, 等. VR环境下基于Leap Motion的三维模型空间位姿自适应调整方法[J]. 计算机应用与软件, 2021, 38(7): 67-72. doi: 10.3969/j.issn.1000-386x.2021.07.012 BAO W Y, HU J Y, HUANG P D, et al. Leap motion-based adaptive adjustment method for space posture of 3d models in VR environment[J]. Computer Applications and Software, 2021, 38(7): 67-72. (in Chinese) doi: 10.3969/j.issn.1000-386x.2021.07.012
[9]	熊巍, 王清辉, 李静蓉. 面向虚拟装配的层次化交互手势技术[J]. 华南理工大学学报(自然科学版), 2016, 44(1): 78-84. XIONG W, WANG Q H, LI J R. A hierarchical method of interaction hand gesture for virtual assembly[J]. Journal of South China University of Technology (Natural Science Edition), 2016, 44(1): 78-84. (in Chinese)
[10]	袁伟, 郎璞, 张孙嘉, 等. 虚拟环境下面向装配培训的人机交互技术[J]. 兵器装备工程学报, 2018, 39(12): 187-191. YUAN W, LANG P, ZHANG S J, et al. Human-computer interaction technology for assembly training under virtual reality[J]. Journal of Ordnance Equipment Engineering, 2018, 39(12): 187-191. (in Chinese)
[11]	WANG X, ONG S K, NEE A Y C. Real-virtual components interaction for assembly simulation and planning[J]. Robotics and Computer-Integrated Manufacturing, 2016, 41: 102-114. doi: 10.1016/j.rcim.2016.03.005
[12]	LIANG J B, HE H W, WU Y M. Bare-Hand depth perception used in augmented reality assembly supporting[J]. IEEE Access, 2020, 8: 1534-1541. doi: 10.1109/ACCESS.2019.2962112
[13]	同济大学数学系. 高等数学[M]. 7版. 北京: 高等教育出版社, 2014. Department of Mathematics, Tongji University. Advanced mathematics[M]. 7th ed. Beijing: Higher Education Press, 2014. (in Chinese)
[14]	HEARN D, BAKER M P, CARITHERS W R. Computer graphics with open GL[M]. 4th ed. New Jersey: Pearson, 2010.
[15]	RODRIGUES O. Des lois géométriques qui régissent les déplacements d'un système solide dans l'espace, et de la variation des coordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire[J]. Journal de Mathématiques Pures et Appliquées, 1840, 5: 380-440.