Adopting multiple vision transformer layers for fine-grained image representation

Sun, Fayou and Ngo, Hea Choon and Yu, Yelan and Xiao, Zhengyu and Meng, Zuqiang (2023) Adopting multiple vision transformer layers for fine-grained image representation. In: 1st International Conference on Computer Technology and Information Science, CTIS 2023, 17 June 2023through 18 June 2023, Virtual, Online.

Text Adopting multiple vision transformer layers for fine-grained image representation.pdf Restricted to Repository staff only Download (707kB)

Abstract

Accurate discriminative regions proposal has an important effect for fine-grained image recognition. The vision transformer (ViT) brings about a striking effect in computer vision duo to its innate muti-head self-attention mechanism. However, the attention maps are gradually similar after certain layers and since ViT adds classification token for perform classification, it is unable to effectively select discriminative image patches for fine-grained image classification. To accurately detect discriminative regions, we propose a novel network AMTrans, which efficiently increases layers to learn diverse features and utilizes integrated raw attention maps to capture more salient feature. Specifically, we employ DeepViT as backbone to solve the attention collapse issue. Then, we fuse each head attention weight within each layer to produce attention weight map. After that, we alternatively use recurrent residual refinement blocks to promote salient feature detection and then utilize semantic grouping method to propose the discriminative feature region. A lot of experiments prove that AMTrans acquires the SOTA performance on three widely used fine-grained datasets under the same settings, involving Stanford-Cars, Stanford-Dogs and CUB-200-2011.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Computer vision, Feature extraction, Image recognition, Semantics
Divisions:	Faculty of Information and Communication Technology
Depositing User:	Maizatul Najwa Ahmad
Date Deposited:	09 Oct 2024 17:10
Last Modified:	09 Oct 2024 17:10
URI:	http://eprints.utem.edu.my/id/eprint/27940
Statistic Details:	View Download Statistic

Actions (login required)

View Item