Next Article in Journal
On Completeness and Fixed Point Theorems in Fuzzy Metric Spaces
Next Article in Special Issue
KDTM: Multi-Stage Knowledge Distillation Transfer Model for Long-Tailed DGA Detection
Previous Article in Journal
Community-Aware Evolution Similarity for Link Prediction in Dynamic Social Networks
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

VL-Meta: Vision-Language Models for Multimodal Meta-Learning

Faculty of Applied Sciences, Macao Polytechnic University, Macao 999078, China
*
Author to whom correspondence should be addressed.
Mathematics 2024, 12(2), 286; https://doi.org/10.3390/math12020286
Submission received: 29 December 2023 / Revised: 12 January 2024 / Accepted: 13 January 2024 / Published: 16 January 2024

Abstract

Multimodal learning is a promising area in artificial intelligence (AI) that can make the model understand different kinds of data. Existing works are trying to re-train a new model based on pre-trained models that requires much data, computation power, and time. However, it is difficult to achieve in low-resource or small-sample situations. Therefore, we propose VL-Meta, Vision Language Models for Multimodal Meta Learning. It (1) presents the vision-language mapper and multimodal fusion mapper, which are light model structures, to use the existing pre-trained models to make models understand images to language feature space and save training data, computation power, and time; (2) constructs the meta-task pool that can only use a small amount of data to construct enough training data and improve the generalization of the model to learn the data knowledge and task knowledge; (3) proposes the token-level training that can align inputs with the outputs during training to improve the model performance; and (4) adopts the multi-task fusion loss to learn the different abilities for the models. It achieves a good performance on the Visual Question Answering (VQA) task, which shows the feasibility and effectiveness of the model. This solution can help blind or visually impaired individuals obtain visual information.
Keywords: vision-language models; multimodal learning; meta-learning; token-level training; visual question answering vision-language models; multimodal learning; meta-learning; token-level training; visual question answering

Share and Cite

MDPI and ACS Style

Ma, H.; Fan, B.; Ng, B.K.; Lam, C.-T. VL-Meta: Vision-Language Models for Multimodal Meta-Learning. Mathematics 2024, 12, 286. https://doi.org/10.3390/math12020286

AMA Style

Ma H, Fan B, Ng BK, Lam C-T. VL-Meta: Vision-Language Models for Multimodal Meta-Learning. Mathematics. 2024; 12(2):286. https://doi.org/10.3390/math12020286

Chicago/Turabian Style

Ma, Han, Baoyu Fan, Benjamin K. Ng, and Chan-Tong Lam. 2024. "VL-Meta: Vision-Language Models for Multimodal Meta-Learning" Mathematics 12, no. 2: 286. https://doi.org/10.3390/math12020286

APA Style

Ma, H., Fan, B., Ng, B. K., & Lam, C.-T. (2024). VL-Meta: Vision-Language Models for Multimodal Meta-Learning. Mathematics, 12(2), 286. https://doi.org/10.3390/math12020286

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop