A method for fabric defect detection based on improved cascade R-CNN

doi:10.19398/j.att.202106018

Abstract

Abstract:

To solve the difficult detection problem due to the uneven distribution of different fabric defects, extreme aspect ratios existing in some defects, and a large number of small targets, a method for fabric defect detection based on improved cascade R-CNN was proposed. The difficult examples were mined online in R-CNN part to strengthen small target training. To address the issue of the extreme aspect ratio of fabric defects, the traditional square volume in the feature extraction network was replaced by deformable convolution v2. The scale of the bounding box was redesigned according to the characteristics of the fabric. Finally, the complete intersection over union loss was adopted as the bounding box regression loss, and a more accurate target bounding box was obtained. The experimental results indicated that the improved model was more accurate in predicting the bounding box than that before improvement, and it achieved a better effect on small target detection. The accuracy was improved by 3.57%, and the average accuracy was improved by 6.45%. Therefore, it can better meet the requirements of fabric defect detection.

Key words: cascade R-CNN, fabric defect, detection, deformable convolution v2, online difficult example mining, complete intersection over union loss

摘要：

由于布匹疵点种类分布不均,部分疵点具有极端的宽高比,而且小目标较多,导致检测难度大,因此提出一种改进级联R-CNN的布匹疵点检测方法。针对小目标问题,在R-CNN部分采用在线难例挖掘,加强对小目标的训练;针对布匹疵点极端的长宽比,在特征提取网络中采用了可变形卷积v2来代替传统的正方形卷积,并结合布匹特征重新设计边界框比例。最后采用完全交并比损失作为边界框回归损失,获取更精确的目标边界框。结果表明:对比改进前的模型,改进后的模型预测边界框更加精确,对小目标的疵点检测效果更好,在准确率上提升了3.57%,平均精确度均值提升了6.45%,可以更好地满足面料疵点的检测需求。

关键词: 级联R-CNN, 面料疵点, 检测, 可变形卷积v2, 在线难例挖掘, 完全交并比损失

CLC Number:

XU Shengbao, ZHENG Liaomo, YUAN Decheng. A method for fabric defect detection based on improved cascade R-CNN[J]. Advanced Textile Technology, 2022, 30(2): 48-56.

许胜宝, 郑飂默, 袁德成. 基于改进级联R-CNN的面料疵点检测方法[J]. 现代纺织技术, 2022, 30(2): 48-56.

Figures/Tables 14

References 23

[1]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: Common objects in context[C]// European Conference on Computer Vision, Cham: Pringer International Pubcishing, 2014:740-755.
[2]	HU X, XU X, XIAO Y, et al. SINet: A scale-insensitive convolutional neural network for fast vehicle detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(3):1010-1019. DOI URL
[3]	LI J, LIANG X, WEI Y, et al. Perceptual generative adversarial networks for small object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA. IEEE, 2017: 1222-1230.
[4]	陈康, 朱威, 任振峰, 等. 基于深度残差网络的布匹疵点检测方法[J]. 小型微型计算机系统, 2020, 41(4):800-806.
	CHEN Kang, ZHU Wei, REN Zhenfeng, et al. Fabric defect detection method based on deep residual network[J]. Journal of Chinese Mini-Micro Computer Systems. 2020, 41(4):800-806.
[5]	孟志青, 邱健数. 基于级联卷积神经网络的复杂花色布匹瑕疵检测算法[J]. 模式识别与人工智能, 2020, 33(12):1135-1144.
	MENG Zhiqing, QIU Jianshu. Defect detection algorithm of complex pattern fabric based on cascaded convolution neural network[J]. Pattern Recognition and Artificial Intelligence, 2020, 33(12):1135-1144.
[6]	CHAWLA N V, BOWYER K W, HALL L O, et al. Synthetic minority over-sampling technique[J]. Journal of artificial intelligence research, 2002, 16:321-357. DOI URL
[7]	YANG Y Z, XU Z. Rethinking the value of labels for improving class-imbalanced learning[EB/OL]. 2020: arXiv: 2006.07529[cs.LG]. https://arxiv.org/abs/2006.07529.
[8]	CAI Z W, VASCONCELOS N. Cascade R-CNN: Delving into high quality object detection[C]// Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA. IEEE, 2018: 6154-6162.
[9]	ZHU X Z, HU H, LIN S, et al. Deformable ConvNets V2: More deformable, better results[C]// Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA. IEEE, 2019: 9300-9308.
[10]	SHRIVASTAVA A, GUPTA A, GIRSHICK R. Training region-based object detectors with online hard example mining[C]// Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA. IEEE, 2016: 761-769.
[11]	ZHENG Z H, WANG P, LIU W, et al. Distance-IoU loss: Faster and better learning for bounding box regression[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7):12993-13000. DOI URL
[12]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. DOI URL
[13]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA. IEEE, 2016: 770-778.
[14]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA. IEEE, 2017: 936-944.
[15]	DAI J F, QI H Z, XIONG Y W, et al. Deformable convolutional networks[C]// International Conference on Computer Vision. Venice, Italy. IEEE, 2017: 764-773.
[16]	YU J H, JIANG Y N, WANG Z Y, et al. UnitBox: An advanced object detection network[C]// Proceedings of the 24th ACM International Conference on Multimedia. Amsterdam The Netherlands. New York, NY, USA: ACM, 2016: 516-520.
[17]	REZATOFIGHI H, TSOI N, GWAK J, et al. Generalized intersection over union: A metric and a loss for bounding box regression[C]// Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA. IEEE, 2019: 658-666.
[18]	BODLA N, SINGH B, CHELLAPPA R, et al. Soft-NMS: Improving object detection with one line of code[C]// International Conference on Computer Vision. Venice, Italy. IEEE, 2017: 5562-5570.
[19]	NEUBECK A, VAN GOOL L. Efficient non-maximum suppression[C]// 18th International Conference on Pattern Recognition. Hong Kong, China. IEEE, 2006: 850-855.
[20]	安萌, 郑飂默, 王诗宇, 等. 一种改进Faster R-CNN的面料疵点检测方法[J]. 小型微型计算机系统, 2021, 42(5):1029-1033.
	AN Meng, ZHENG Liaomo, WANG Shiyu, et al. A fabric defect detection method based on improved faster R-CNN[J]. Journal of Chinese Computer Systems, 2021, 42(5):1029-1033.
[21]	张泽苗, 霍欢, 赵逢禹. 深层卷积神经网络的目标检测算法综述[J]. 小型微型计算机系统, 2019, 40(9):1825-1831.
	ZHANG Zemiao, HUO Huan, ZHAO Fengyu. Survey of object detection algorithm based on deep convolutional neural networks[J]. Journal of Chinese Computer Systems, 2019, 40(9):1825-1831.
[22]	PADILLA R, PASSOS W L, DIAS T L B, et al. A comparative analysis of object detection metrics with a companion open-source toolkit[J]. Electronics, 2021, 10(3):279. DOI URL
[23]	陈科圻, 朱志亮, 邓小明, 等. 多尺度目标检测的深度学习研究综述[J]. 软件学报, 2021, 32(4):1201-1227.
	CHEN Keqi, ZHU Zhiliang, DENG Xiaoming, et al. Deep learning for multi-scale object detection: asurvey[J]. Journal of Software. 2021, 32(4):1201-1227.

编号	瑕疵	图片数/张	目标数/个	编号	瑕疵	图片数/张	目标数/个
1	破洞	150	312	11	吊经	147	160
2	水渍、油渍、污渍	259	401	12	粗纬	545	915
3	三丝	845	1055	13	纬缩	262	452
4	结头	1447	1996	14	浆斑	284	336
5	花板跳	122	134	15	整经结	186	473
6	百脚	140	161	16	星跳、跳花	295	302
7	毛粒	160	195	17	断氨纶	222	546
8	粗经	205	213	18	稀密档、浪纹档、色差档	354	361
9	松经	359	396	19	磨痕、轧痕、修痕、烧毛痕	415	470
10	断经	265	284	20	死皱、云织、双纬、双经、跳纱、筘路、纬纱不良	289	361

编号	瑕疵	图片数/张	目标数/个	编号	瑕疵	图片数/张	目标数/个
1	破洞	150	312	11	吊经	147	160
2	水渍、油渍、污渍	259	401	12	粗纬	545	915
3	三丝	845	1055	13	纬缩	262	452
4	结头	1447	1996	14	浆斑	284	336
5	花板跳	122	134	15	整经结	186	473
6	百脚	140	161	16	星跳、跳花	295	302
7	毛粒	160	195	17	断氨纶	222	546
8	粗经	205	213	18	稀密档、浪纹档、色差档	354	361
9	松经	359	396	19	磨痕、轧痕、修痕、烧毛痕	415	470
10	断经	265	284	20	死皱、云织、双纬、双经、跳纱、筘路、纬纱不良	289	361

疵点编号	算法
疵点编号	Pre(Cascade R-CNN)	Pre+OHEM	Pre+OHEM+DCN v2	Pre+OHEM+DCN v2+CIoU Loss
1	68.4	69.1	71.0	73.4
2	33.9	37.6	36.8	38.9
3	70.6	71.4	73.4	73.6
4	45.4	46.7	50.0	51.4
5	82.9	83.6	86.6	90.9
6	42.6	46.8	53.2	58.6
7	29.3	34.6	38.9	38.3
8	57.7	61.1	65.7	65.7
9	63.9	64.4	69.9	70.9
10	43.1	46.6	51.3	52.0
11	54.6	54.8	55.7	55.6
12	49.9	51.2	51.4	54.9
13	31.8	37.3	39.8	40.8
14	80.2	79.9	82.2	83.2
15	30.2	34.9	32.5	30.3
16	50.4	52.3	58.2	58.4
17	37.6	41.9	44.1	49.6
18	28.6	33.2	29.6	30.6
19	40.9	42.8	44.9	44.9
20	34.2	39.6	40.4	43.2
ACC	93.63	92.87	96.47	97.20
mAP	48.81	51.49	53.78	55.26

疵点编号	算法
疵点编号	Pre(Cascade R-CNN)	Pre+OHEM	Pre+OHEM+DCN v2	Pre+OHEM+DCN v2+CIoU Loss
1	68.4	69.1	71.0	73.4
2	33.9	37.6	36.8	38.9
3	70.6	71.4	73.4	73.6
4	45.4	46.7	50.0	51.4
5	82.9	83.6	86.6	90.9
6	42.6	46.8	53.2	58.6
7	29.3	34.6	38.9	38.3
8	57.7	61.1	65.7	65.7
9	63.9	64.4	69.9	70.9
10	43.1	46.6	51.3	52.0
11	54.6	54.8	55.7	55.6
12	49.9	51.2	51.4	54.9
13	31.8	37.3	39.8	40.8
14	80.2	79.9	82.2	83.2
15	30.2	34.9	32.5	30.3
16	50.4	52.3	58.2	58.4
17	37.6	41.9	44.1	49.6
18	28.6	33.2	29.6	30.6
19	40.9	42.8	44.9	44.9
20	34.2	39.6	40.4	43.2
ACC	93.63	92.87	96.47	97.20
mAP	48.81	51.49	53.78	55.26

光照强度	ACC/%	mAP/%
3.0	96.00	49.83
4.0	96.27	49.35
4.5	96.67	50.84
5.0	96.83	54.87
5.5	97.27	51.78
6.0	97.27	50.95
7.0	96.73	49.66
10.0	94.93	48.05
未设置强度	97.20	55.26