2026-05-13T19:57:23Z https://ipsj.ixsq.nii.ac.jp/oai

oai:ipsj.ixsq.nii.ac.jp:00235609 2025-01-19T09:35:36Z 1164:2735:11468:11702

Vision Transformer-based Meta Loss Landscape Exploration with Actor-Critic Method Vision Transformer-based Meta Loss Landscape Exploration with Actor-Critic Method Enzhi, Zhang Xuanyi, Wu Rui, Zhong Xingbang, Du Mohamed, Wahib Masaharu, Munetomo Enzhi, Zhang Xuanyi, Wu Rui, Zhong Xingbang, Du Mohamed, Wahib Masaharu, Munetomo Detecting and mitigating overfitting in deep neural networks remains a critical challenge in modern machine learning. This paper investigates innovative approaches to address these challenges, particularly focusing on vision transformer-based models. By leveraging meta-learning techniques and reinforcement learning frameworks, we introduce Transformer-based Loss Landscape Exploration (TLLE), which utilizes the validation loss landscape to guide gradient descent optimization. Unlike conventional methods, TLLE employs the Actor-Critic algorithm to learn the mapping from model weights to future values, facilitating efficient sample collection and precise value predictions. Experimental results demonstrate the superior performance of TLLE-enhanced transformer models in image classification and segmentation tasks, showcasing the efficacy of our approach in optimizing deep learning models for image analysis. Detecting and mitigating overfitting in deep neural networks remains a critical challenge in modern machine learning. This paper investigates innovative approaches to address these challenges, particularly focusing on vision transformer-based models. By leveraging meta-learning techniques and reinforcement learning frameworks, we introduce Transformer-based Loss Landscape Exploration (TLLE), which utilizes the validation loss landscape to guide gradient descent optimization. Unlike conventional methods, TLLE employs the Actor-Critic algorithm to learn the mapping from model weights to future values, facilitating efficient sample collection and precise value predictions. Experimental results demonstrate the superior performance of TLLE-enhanced transformer models in image classification and segmentation tasks, showcasing the efficacy of our approach in optimizing deep learning models for image analysis. technical report 情報処理学会 2024-07-15 application/pdf 研究報告数理モデル化と問題解決（MPS） 12 2024-MPS-149 1 5 2188-8833 AN10505667 https://ipsj.ixsq.nii.ac.jp/record/235609/files/IPSJ-MPS24149012.pdf eng