|
|
论文:2022,Vol:40,Issue(4):804-811 |
|
|
引用本文： |
|
|
薛峰, 武君胜, 张涛, 王威, 成静. 面向移动应用自动化测试的同构用户界面视觉判断方法[J]. 西北工业大学学报 |
|
|
XUE Feng, WU Junsheng, ZHANG Tao, WANG Wei, CHENG Jing. Visual judgment approach of isomorphic GUI for automated mobile app testing[J]. Northwestern polytechnical university |
|
|
|
|
|
|
|
| 面向移动应用自动化测试的同构用户界面视觉判断方法 |
|
| 薛峰1, 武君胜2, 张涛2, 王威2, 成静3 |
|
1. 西北工业大学 计算机学院, 陕西 西安 710072;
2. 西北工业大学 软件学院, 陕西 西安 710072;
3. 西安工业大学 计算机科学与工程学院, 陕西 西安 710021 |
| 摘要: |
| 当前快速增长的移动应用程序迫切需求自动化测试技术以保证其质量。移动应用的自动化测试与其图形用户界面(GUI)的识别与判断紧密相关,但移动应用却通常存在大量的样式与内容有差异而结构与功能相类似的同构GUI。在自动化测试中,同构GUI容易引发应用状态空间的爆炸问题,进而导致测试的低效或失败。针对传统自动化识别同构GUI的局限性,提出一种基于视觉特征信息的GUI相似度判断方法。通过目标检测技术识别GUI组件元素进而抽象出GUI结构框架;采用卷积自编码器提取出GUI结构视觉特征;对比GUI视觉特征的相似度完成同构GUI判断。经过实验验证,所提方法能够屏蔽GUI的样式、内容等影响,从而更精确地完成同构GUI识别,优化自动化测试效率。 |
| 关键词:
移动应用测试
GUI视觉特征
同构GUI
GUI相似判断
|
|
| Visual judgment approach of isomorphic GUI for automated mobile app testing |
|
| XUE Feng1, WU Junsheng2, ZHANG Tao2, WANG Wei2, CHENG Jing3 |
|
1. School of Computer Science, Northwestern Polytechnical University, Xi'an 710072, China;
2. School of Software, Northwestern Polytechnical University, Xi'an 710072, China;
3. School of Computer Science and Engineering, Xi'an Technological University, Xi'an 710021, China |
| Abstract: |
| Currently, the rapid growth of mobile apps requires automated testing technology to ensure their quality. Automated testing of mobile apps is usually closely related to the recognition and judgment of their graphical user interface (GUI), but there usually are numerous isomorphic GUIs with different styles and contents, and similar structure and function in mobile apps. In automatic testing, isomorphic GUI is easy to cause the issue of state space explosion, which leads to low efficiency or failure of testing. In view of the limitations of traditional automatic recognition of isomorphic GUI, this paper presents a GUI similarity judgment approach based on visual feature information. Firstly, the GUI component elements are identified by object detection, and then the GUI skeleton is abstracted. Secondly, the visual features of the GUI skeleton are extracted by a convolutional autoencoder. Finally, the isomorphic GUI judgment is completed by comparing the similarity of GUI visual features. The experimental results show that the proposed approach can effectively shield the influence of GUI style and content, complete the isomorphic GUI recognition more accurately and optimize the efficiency of automated mobile app testing. |
| Key words:
mobile app testing
GUI visual features
isomorphic GUI
GUI similarity judgment
|
|
| 收稿日期: 2021-09-15
修回日期:
|
| DOI: 10.1051/jnwpu/20224040804 |
| 通讯作者: 张涛(1976-),西北工业大学副教授,主要从事软件工程、软件系统架构研究。e-mail:tao_zhang@nwpu.edu.cn
Email：tao_zhang@nwpu.edu.cn |
| 作者简介: 薛峰(1987-),西北工业大学博士研究生,主要从事软件工程、软件测试研究。
|
|
| 相关功能 |
|
|
|
|
| 作者相关文章 |
|
| 薛峰 在本刊中的所有文章 |
| 武君胜 在本刊中的所有文章 |
| 张涛 在本刊中的所有文章 |
| 王威 在本刊中的所有文章 |
| 成静 在本刊中的所有文章 |
|
|
|
|
|
|
|
|
参考文献: |
|
|
[1] KONG P, LI L, GAO J, et al. Automated testing of Android apps: a systematic literature review[J]. IEEE Trans on Reliability, 2018, 68(1): 45-66
[2] TRAMONTANA P, AMALFITANO D, AMATUCCI N, et al. Automated functional testing of mobile applications: a systematic mapping study[J]. Software Quality Journal, 2019, 27(1): 149-201
[3] LINARES-VÁSQUEZ M, BERNAL-CÁRDENAS C, MORAN K, et al. How do developers test android applications?[C]//2017 IEEE International Conference on Software Maintenance and Evolution, 2017: 613-622
[4] RUBINOV K, BARESI L. What are we missing when testing our android apps?[J]. Computer, 2018, 51(4): 60-68
[5] Appium-automation for apps[EB/OL].(2018-10-05)[2021-08-18]. http://appium.io/dols/cn/about-appium/intro
[6] Eyeautomate-visual script runner[EB/OL].(2019-02-08)[2021-08-12]. https://eyeautomate.com/eyeautomate
[7] AMALFITANO D, RICCIO V, AMATUCCI N, et al. Combining automated GUI exploration of android apps with capture and replay through machine learning[J]. Information and Software Technology, 2019, 105: 95-116
[8] GUO J, LI S, LOU J G, et al. SARA: self-replay augmented record and replay for android in industrial cases[C]//Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2019: 90-100
[9] GU T, SUN C, MA X, et al. Practical GUI testing of Android applications via model abstraction and refinement[C]//2019 IEEE/ACM 41st International Conference on Software Engineering, 2019: 269-280
[10] SALIHU I A, IBRAHIM R, AHMED B S, et al. AMOGA: a static-dynamic model generation strategy for mobile apps testing[J]. IEEE Access, 2019, 7: 17158-17173
[11] BEHRANG F, ORSO A. Test migration between mobile apps with similar functionality[C]//2019 34th IEEE/ACM International Conference on Automated Software Engineering, 2019: 54-65
[12] PAN M, XU T, PEI Y, et al. GUI-guided test script repair for mobile apps[J]. IEEE Trans on Software Engineering, 2022, 48(3): 910-929
[13] CRACIUNESCU M, MOCANU S, DOBRE C, et al. Robot based automated testing procedure dedicated to mobile devices[C]//2018 25th International Conference on Systems, Signals and Image Processing, 2018: 1-4
[14] MAO K, HARMAN M, JIA Y. Robotic testing of mobile APPS for truly black-box automation[J]. IEEE Software, 2017, 34(2): 11-16
[15] BANERJEE D, YU K. Robotic arm-based face recognition software test automation[J]. IEEE Access, 2018, 6: 37858-37868
[16] NASS M, ALÉGROTH E, FELDT R. Why many challenges with GUI test automation(will) remain[J]. Information and Software Technology, 2021, 138: 106625
[17] DEKA B, HUANG Z, FRANZEN C, et al. RICO: a mobile app dataset for building data-driven design applications[C]//Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, 2017: 845-854
[18] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection[J/OL].(2020-04-05)[2021-08-12]. https://arxiv.org/abs/2004.10934
[19] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//European Conference on Computer Vision, Cham, 2016: 21-37
[20] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2980-2988
[21] CHEN C, SU T, MENG G, et al. From UI design image to GUI skeleton: a neural machine translator to bootstrap mobile GUI implementation[C]//Proceedings of the 40th International Conference on Software Engineering. 2018: 665-676
[22] MORAN K, BERNAL-CÁRDENAS C, CURCIO M, et al. Machine learning-based prototyping of graphical user interfaces for mobile apps[J]. IEEE Trans on Software Engineering, 2018, 46(2): 196-221
[23] MASCI J, MEIER U, CIRESAN D, et al. Stacked convolutional auto-encoders for hierarchical feature extraction[C]//International Conference on Artificial Neural Networks, Berlin, Heidelberg, 2011: 52-59 |
|
|
|
|
|
|
|
