基于CycleGAN的服装图像混搭风格迁移

摘要/Abstract

摘要： 为解决复杂背景下服装图像的风格迁移形式单一和局部细节失真问题，提出一种基于CycleGAN的服装图像混搭风格迁移的方法，用于实现服装款式和图案的多风格迁移。通过加入分割掩码，一方面，对特定区域的风格化形成空间约束，在判别器中加入谱归一化和引入背景优化损失保留了局部细节的真实度，实现服装风格款式的风格迁移；另一方面，提出图像融合的方式，将图案融入判别器输出的服装图像中，实现多风格迁移。最后，通过与CycleGAN和InstaGAN比较，依据生成图像的效果进行主观分析，使用图像质量评估指标IS和SSIM进行客观评估以验证该方法的有效性。

关键词: 混搭风格迁移, 生成对抗网络, 服装款式, 服装图案, 智能设计, 服装设计

Abstract: With the continuous integration of artificial intelligence (AI) technology and the fashion field, the use of style transfer technology to generate new images has become one of the research hotspots of aided intelligent clothing design. However, the use of current style transfer technology in the process of aided intelligent design still has great limitations. Only completing the transfer of a single style limits the diversity of generated clothing images, and the detail distortion of the clothing image with the character background reduces the authenticity of the generated clothing images.
Aiming at solving the problems of the undiversified transfer form and local detail distortion of clothing image styles in complicated conditions, a method of clothing image mixing and matching style transfer was proposed to realize the multi-style transfer of clothing styles and patterns. During the experiment, we took CycleGAN as the baseline model, with the advantage of improving the effect of style transfer without requiring pairwise training and cyclic consistency loss, used Resnet generator and PatchGAN discriminator for training, and introduced the segmentation mask. On the one hand, spatial constraints were formed for the stylization of specific areas, instance normalization was added to the discriminator to maintain the independence of image instances, spectral normalization was added to the first and last layers of the convolution layer to enhance the classification ability of the network, and background optimization loss was added to optimize the local details of the generated images, especially the boundary artifacts, which jointly promoted the generation effect and realized the style transfer of clothing styles. On the other hand, the method of image fusion was proposed. According to the pattern fusion mapping relationship, the pattern was integrated into the clothing image output by the discriminator to realize the multi-style transfer of clothing styles and patterns. In order to verify the effectiveness of the above method in the multi-style transfer of clothing images, the clothing image design sketches generated in the experiment were compared with the design sketches generated by CycleGAN and InstaGAN models. By subjectively analyzing the style diversity and detail differences of the design sketches, the IS and SSIM were used for quantitative analysis, and the subjective visual effect and objective numerical comparison both showed the advantages of this experimental method in the diversity and the authenticity of image details.
The multi-style transfer of clothing styles and patterns can provide designers with creative inspiration and shorten the time period required for effect presentation, making AI more suitable for assisting clothing design behavior. In subsequent experiments, we will explore the transfer of more types of clothing styles, so as to achieve a diversified and controllable style transfer method. In addition, emotional elements needed for perceptual design should be added in the process of image style transfer, so as to promote the integration of computational thinking and design thinking of future-oriented design paradigm.

Key words: mix and match style transfer, generative adversarial network, clothing style, clothing pattern, intelligent design, clothing design

中图分类号:

TS941.26

王伟珍, 张功. 基于CycleGAN的服装图像混搭风格迁移[J]. 现代纺织技术, 2023, 31(4): 250-258.

WANG Weizhena, b, ZHANG Gonga. Mix and match style transfer for the images of clothes with CycleGAN#br#[J]. Advanced Textile Technology, 2023, 31(4): 250-258.

参考文献

[1] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[2] CHENG W H, SONG S J, CHEN C Y, et al. Fashion meets computer vision: A survey[J]. ACM Computing Surveys , 2022, 54(4): 1-41.
[3] 施倩, 罗戎蕾. 基于生成对抗网络的服装图像生成研究进展[J/OL].现代纺织技术, 2022，31: 19-29. [2023-01-23]. https://kns.cnki.net. DOI:10.19398/j.att.202203056.
SHI Qian, LUO Ronglei. Research progress of clothing image generation based on generative adversarial networks[J/OL]. Advanced Textile Technology. 2022,31:19-29[2023-01-23]. https://kns.cnki.net.DOI:10.19398/j.att.202203056.
[4] CHEN H, LEI S, ZHANG S G, et al. Man-algorithm cooperation intelligent design of clothing products in multi links[J]. Fibres and Textiles in Eastern Europe. 2022,30(1): 59-66.
[5] 赵梦如. 人工智能在服装款式设计领域的应用进展[J]. 纺织导报, 2021(12): 74-77.
ZHAO Mengru. Application progress of artificial intelligence in clothing style design[J]. China Textile Leader, 2021(12): 74-77.
[6] JIANG S, LI J, FU Y. Deep learning for fashion style generation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(9): 4538-4550.
[7] 唐稔为, 刘启和, 谭浩. 神经风格迁移模型综述[J]. 计算机工程与应用, 2021, 57(19): 32-43.
TANG Renwei, LIU Qihe, TAN Hao. Review of neural style transfer models[J]. Computer Engineering and Applications, 2021, 57(19): 32-44.
[8] JING Y C, YANG Y Z, FENG Z L, et al. Neural style transfer: A review[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(11): 3365-3385.
[9] MO D, ZOU X, WONG W K. Neural stylist: Towards online styling service[J]. Expert Systems with Applications, 2022, 203: 117333.
[10] GATYS L, ECKER A, BETHGE M. A neural algorithm of artistic style[J]. Journal of Vision, 2016, 16(12): 356.
[11] WANG H Y, XIONG H T, CAI Y Y. Image localized style transfer to design clothes based on CNN and interactive segmentation[J]. Computational Intelligence and Neuroscience, 2020, 2020: 8894309.
[12] LI Y J, FANG C, YANG J M, et al. Universal style transfer via feature transforms[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, California, USA. New York: ACM, 2017: 385-395.
[13] GOODFELLOW I, POUGET-ABADIE J, MIRZA M B, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems:Volume 2. Montreal, Canada. New York: ACM, 2014: 2672-2680.
[14] ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 5967-5976.
[15] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//IEEE International Conference on Computer Vision (ICCV). Venice, Italy. IEEE, 2017: 2242-2251.
[16] MO S, CHO M, SHIN J. InstaGAN: Instance-aware Image-to-Image Translation[J]. ArXiv, 2018: 1812.10889. https:// https://arxiv.org/abs/1812.10889.
[17] SALIMANS T, GOODFELLOW I, ZAREMBA W, et al. Improved techniques for training GANs[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain. New York: ACM, 2016: 2234–2242.
[18] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2004, 13(4): 600-612.
[19] YI Z, ZHANG H, TAN P, et al.DualGAN: Unsupervised dual learning for image-to-image translation[C]// Proceedings of the IEEE International Conference on Computer Vision(ICCV). Venice, Italy. IEEE, 2017: 2868-2876.
[20] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Las Vegas, NV, USA. IEEE, 2016: 770-778.
[21] 林泓, 任硕, 杨益, 等. 融合自注意力机制和相对鉴别的无监督图像翻译[J]. 自动化学报, 2021, 47(9): 2226-2237.
LIN Hong, REN Shuo, YANG Yi, et al. Unsupervised image-to-image translation with self-attention and relativistic discriminator adversarial networks[J]. ACTA Automatica Sinica, 2021, 47(9): 2226-2237.
[22] GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of wasserstein gans[J]. ArXiv, 2017:1704. 00028. https://arxiv.org/abs/1704.00028.
[23] LIANG X D, LIN L, YANG W, et al. Clothes Co-parsing via joint image segmentation and labeling with application to clothing retrieval[J]. IEEE Transactions on Multimedia, 2016, 18(6): 1175-1186.
[24] 陈淮源, 张广驰, 陈高, 等. 基于深度学习的图像风格迁移研究进展[J]. 计算机工程与应用, 2021, 57(11): 37-45.
CHEN Huaiyuan, ZHANG Guangchi, CHEN Gao, et al. Research progress of image style transfer based on depth learning[J]. Computer Engineering and Applications, 2021, 57(11): 37-45.
[25] 李敏, 刘冰清, 彭庆龙, 等. 基于CycleGAN算法的迷彩服装图案设计方法研究[J]. 丝绸, 2022, 59(8): 100-106.
LI Min, LIU Bingqing, PENG Qinglong, et al. A camouflage suit pattern design based on the CycleGAN algorithm[J]. Journal of Silk, 2022, 59(8): 100-106.
[26] 曾宪华, 陆宇喆, 童世玥, 等. 结合马尔科夫场和格拉姆矩阵特征的写实类图像风格迁移[J]. 南京大学学报(自然科学), 2021, 57(1): 1-9.
ZENG Xianhua, LU Yuzhe, TONG Shiyue, et al. Photorealism style transfer combining MRFs⁃based and gram⁃based features[J]. Journal of Nanjing University (Natural Science), 2021, 57(1): 1-9.
[27] QUINON P. Engineered emotions[J]. Science, 2017, 358(6364): 729.