Study on Surface Defect Detection of Metal Sheet and Strip using Faster R-CNN with Multilevel Feature
-
摘要: 针对金属板带材表面缺陷呈现形式存在多样性和随机性而导致难以快速定位并准确识别的问题，提出一种融合多层次特征的Faster R-CNN缺陷目标检测算法（Defect-target detection network, DDN）。该算法采用多层次特征融合网络（Multilevel-feature fusion network, MFN）融合Faster R-CNN中VGG-16提取的各层次特征图，得到具有丰富位置信息和语义信息的融合特征图，后续网络基于该融合特征图产生最终的缺陷检测结果。利用钢带和铜板表面缺陷检测数据集评估本文算法性能，实验结果表明，提出的DNN能够快速准确检测出具有不同尺度的多类缺陷，与Faster R-CNN相比，在不损耗过多检测时间的前提下具有更优的检测精度，平均检测时间为129.65 ms或153.17 ms，平均准确率均值（Mean average precision, mAP）为86.13%或92.54%。
-
关键词:
- 金属板带材 /
- 表面缺陷检测 /
- 准确定位 /
- 多层次特征 /
- Faster R-CNN
Abstract: Aiming at the diversity and randomness of the surface defects of the metal sheet and strip, which makes it difficult to quickly locate and accurately identify, a Defect-Target Detection Network (DNN) by using Faster R-CNN with multilevel feature is proposed. This algorithm uses a Multilevel-Feature Fusion Network (MFN) to fuse the feature maps extracted from VGG-16 in Faster R-CNN to obtain fusion feature maps with rich location information and semantic information. The subsequent networks generate the final defect detection results by using the fusion feature maps. The performance of the present algorithm is evaluated by using the surface defect detection data sets of the steel strip and copper sheet. The experimental results show that the present DNN can detect multiple types of defects with different scales quickly and accurately. Comparing with Faster R-CNN, it has better detection accuracy without losing the basic requirement of excessive detection time speed. The average detection time is of 129.65 ms or 153.17 ms and the mean average precision (mAP) is of 86.13% or 92.54%. -
表 1 MFN各分支网络配置
F2 F3 F4 F5 [3×3,256,s2] [3×3,256,s2] [3×3,256,s2] [1×1,128] [3×3,256,s2] [3×3,256,s2] [1×1,128] − [3×3,256,s2] [1×1,128] − − [1×1,128] − − − 注：s2表示步长为2 
下载: 导出CSV
表 3 NEU-DET数据集中的缺陷样本分布
样本类型 训练样本 验证样本 测试样本 标签 总计 网纹 218 29 53 1 300 夹杂 223 20 57 2 300 斑块 207 27 66 3 300 表面麻点 218 24 58 4 300 氧化铁皮压入 221 17 62 5 300 划伤 209 27 64 6 300 总计 1296 144 360 − 1800
下载: 导出CSV
表 4 KMUST-DET数据集中的缺陷样本分布
样本类型 训练样本 验证样本 测试样本 标签 总计 黑点 362 40 106 1 508 油滴 408 46 116 2 570 划伤 457 51 119 3 627 总计 1227 137 341 − 1705
下载: 导出CSV
表 5 实验计算机主要配置
硬件环境 软件环境 处理器：Intel Xeon CPU
E5-2620 v4开发语言：Python 3.5.2 显卡：NVIDIA 1080 Ti,
12 GB GDDR5X深度学习框架：
tensorflow-gpu-1.6.0内存：32 GB DDR4 操作系统：Windows
Server 2012 R2
下载: 导出CSV
表 6 不同算法在NEU-DET测试集下的测试结果
算法 mAP AP/% 平均测试时间/ms Cr In Pa PS RS Sc Faster R-CNN+VGG-16 85.17 77.79 84.08 90.64 87.71 79.96 90.82 125.48 Faster R-CNN+ResNet-50 84.96 71.47 84.07 88.63 87.84 82.52 95.22 112.33 DNN 86.13 75.11 84.20 88.83 87.76 83.63 97.26 129.65
下载: 导出CSV
表 7 不同算法在KMUST-DET测试集下的测试结果
算法 mAP AP/% 平均测试时间/ms BS OD Sc Faster R-CNN+VGG-16 88.97 90.91 86.12 89.89 143.55 Faster R-CNN+ResNet-50 88.69 90.91 86.04 89.13 128.63 DNN 92.54 99.46 87.86 90.29 153.17
下载: 导出CSV
表 8 不同算法在不同IoU阈值下的mAP
算法 数据集 mAP/% IoU = 0.1 IoU = 0.2 IoU = 0.3 IoU = 0.4 IoU = 0.5 IoU = 0.6 IoU = 0.7 IoU = 0.8 Faster R-CNN+VGG-16 NEU-DET 87.11 86.22 85.17 81.01 70.61 55.19 36.58 14.97 Faster R-CNN+ResNet-50 88.01 86.75 84.96 81.41 70.25 53.30 33.27 15.75 DNN 88.72 88.62 86.13 80.40 71.79 57.01 37.50 17.14 Faster R-CNN+VGG-16 KMUST-DET 92.38 92.08 88.97 86.89 84.01 76.11 62.01 38.15 Faster R-CNN+ResNet-50 91.44 91.20 88.69 86.38 84.47 74.25 56.13 26.58 DNN 92.79 92.71 92.53 89.06 85.85 78.20 63.11 33.11
下载: 导出CSV
表 9 不同算法在不同候选区域数目K下的mAP
算法 数据集 mAP/% K = 50 K = 100 K = 150 K = 200 K = 250 K = 300 Faster R-CNN+VGG-16 NEU-DET 83.65 85.04 84.62 84.74 85.33 85.17 Faster R-CNN+ResNet-50 82.62 85.00 85.33 85.60 85.27 84.96 DNN 83.47 84.91 85.03 86.12 86.47 86.13 Faster R-CNN+VGG-16 KMUST-DET 86.67 89.74 90.35 88.89 89.06 88.97 Faster R-CNN+ResNet-50 83.96 86.86 88.71 88.61 88.74 88.69 DNN 84.00 91.47 92.44 92.46 92.49 92.53
下载: 导出CSV
-
[1] GHORAI S, MUKHERJEE A, GANGADARAN M, et al. Automatic defect detection on hot-rolled flat steel products[J]. IEEE Transactions on Instrumentation and Measurement, 2013, 62(3): 612-621 doi: 10.1109/TIM.2012.2218677 [2] XIAO M, JIANG M M, LI G Y, et al. An evolutionary classifier for steel surface defects with small sample set[J]. EURASIP Journal on Image and Video Processing, 2017, 2017(1): 48 doi: 10.1186/s13640-017-0197-y [3] CHU M X, ZHAO J, LIU X P, et al. Multi-class classification for steel surface defects based on machine learning with quantile hyper-spheres[J]. Chemometrics and Intelligent Laboratory Systems, 2017, 168: 15-27 doi: 10.1016/j.chemolab.2017.07.008 [4] 化春键, 周海英. 改进组合分类器的冷轧带钢表面缺陷识别研究[J]. 机械科学与技术, 2017, 36(11): 1785-1790HUA C J, ZHOU H Y. Study on surface defect recognition of cold rolled steel strip by improving combinationclassifier[J]. Mechanical Science and Technology for Aerospace Engineering, 2017, 36(11): 1785-1790 (in Chinese) [5] LIU K, WANG H Y, CHEN H Y, et al. Steel surface defect detection using a new Haar-Weibull-variance model in unsupervised manner[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66(10): 2585-2596 doi: 10.1109/TIM.2017.2712838 [6] YI L, LI G Y, JIANG M M. An end‐to‐end steel strip surface defects recognition system based on convolutional neural networks[J]. Steel Research International, 2017, 88(2): 1600068 doi: 10.1002/srin.201600068 [7] ZHOU S Y, CHEN Y P, ZHANG D L, et al. Classification of surface defects on steel sheet using convolutional neural networks[J]. Materiali in Tehnologije, 2017, 51(1): 123-131 doi: 10.17222/mit.2015.335 [8] NATARAJAN V, HUNG T Y, VAIKUNDAM S, et al. Convolutional networks for voting-based anomaly classification in metal surface inspection[C]// Proceedings of 2017 IEEE International Conference on Industrial Technology. Toronto, ON, Canada: IEEE, 2017 [9] HE D, XU K, ZHOU P, et al. Surface defect classification of steels with a new semi-supervised learning method[J]. Optics and Lasers in Engineering, 2019, 117: 40-48 doi: 10.1016/j.optlaseng.2019.01.011 [10] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 580-587 [11] GIRSHICK R. Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 1440-1448. [12] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149 doi: 10.1109/TPAMI.2016.2577031 [13] LANZA-GUTIÉRREZ J M, XU X J, YANG F, et al. Railway subgrade defect automatic recognition method based on improved faster R-CNN[J]. Scientific Programming, 2018, 2018: 4832972 [14] 常海涛, 苟军年, 李晓梅. Faster R-CNN在工业CT图像缺陷检测中的应用[J]. 中国图象图形学报, 2018, 23(7): 1061-1071 doi: 10.11834/jig.170577CHANG H T, GOU J N, LI X M. Application of faster R-CNN in image defect detection of industrial CT[J]. Journal of Image and Graphics, 2018, 23(7): 1061-1071 (in Chinese) doi: 10.11834/jig.170577 [15] SONG K C, YAN Y H. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects[J]. Applied Surface Science, 2013, 285: 858-864 doi: 10.1016/j.apsusc.2013.09.002 -
点击查看大图
图(5) / 表(9)
计量
- 文章访问数: 828
- HTML全文浏览量: 207
- PDF下载量: 46
- 被引次数: 0
