关键词: 交互式, 多头注意力, StableVITON, 虚拟试穿, 稳定扩散

Abstract: With the booming development of e-commerce and the popularity of online clothing shopping, virtual try-on technology has been significantly promoted. At present, virtual try-on technology is mainly divided into two categories: 3D and 2D images, among which 2D image virtual try-on is widely used due to its easy operation and low cost. This technology is further subdivided into methods based on Generative Adversarial Networks (GANs) and diffusion networks. In recent years, virtual try-on based on diffusion networks has received widespread attention due to its superior performance in realism, stability, and detail processing compared to GAN networks. StableVITON is an important benchmark model in this field and has achieved significant results in synthesizing try-on images by relying on the powerful generation ability of diffusion networks. However, there are still shortcomings in capturing and preserving clothing features and details, such as the inability to accurately identify clothing's long and short sleeves, colors, as well as details such as cuffs and necklines.
To address the problem of clothing feature and detail loss in the StableVITON, this paper proposed a virtual try-on network based on an interactive multi-head attention mechanism. Specifically, this article introduced an interactive multi-head attention mechanism in the clothing encoding block of the StableVITON to facilitate the interaction between different heads and learn rich feature correlations, so as to enhance the network attention performance and retain more clothing features and details. This article adopted various strategies to achieve this goal. Firstly, the latent space of the diffusion network was pre-trained to learn semantic correspondences between clothing and the human body. Secondly,  zero-cross-attention mechanism was introduced into the U-Net decoder. Lastly, the multi-head attention was adjusted to an interactive version which learns rich feature correlations through dense interaction mechanisms to enhance the combination of local and global information, reduce information loss, and improve the learning efficiency and stability of the model.
To verify the effectiveness of the proposed method, qualitative and quantitative experiments were conducted on the VITON-HD dataset. Results show that this virtual try-on network generates more realistic overall clothing features and local details compared to other mainstream models. Compared to StableVITON, it improves the average Structural Similarity Index (SSIM) by 1.53%, reduces the average Learned Perceptual Image Patch Similarity (LPIPS) by 0.71%, lowers Fréchet Inception Distance (FID) by 0.15%, and decreases Kernel Inception Distance (KID) by 1.14%. This network effectively preserves clothing feature details and significantly enhances image fidelity and its synthesized try-on images can provide consumers with a better shopping experience and can be widely used in digital fashion applications such as virtual try-on.

Key words: interactive, multi-head attention, StableVITON, virtual try-on, Stable Diffusion