2021 TransformerisallYouNeedMultimod

• (Hu & Singh, 2021) ⇒ Ronghang Hu, and Amanpreet Singh. (2021). “Transformer is all You Need: Multimodal Multitask Learning with a Unified Transformer.” In: arXiv e-prints.

Subject Headings:

Cited By

• http://scholar.google.com/scholar?q=%222021%22+Transformer+is+all+You+Need%3A+Multimodal+Multitask+Learning+with+a+Unified+Transformer

Quotes

Abstract

We propose UniT, a Unified Transformer model to simultaneously learn the most prominent tasks across different domains, ranging from object detection to language understanding and multimodal reasoning. Based on the transformer encoder-decoder architecture, our UniT model encodes each input modality with an encoder and makes predictions on each task with a shared decoder over the encoded input representations, followed by task-specific output heads. The entire model is jointly trained end-to-end with losses from each task. Compared to previous efforts on multi-task learning with transformers, we share the same model parameters to all tasks instead of separately fine-tuning task-specific models and handle a much higher variety of tasks across different domains. In our experiments, we learn 7 tasks jointly over 8 datasets, achieving comparable performance to well-established prior work on each domain under the same supervision with a compact set of model parameters. Code will be released in MMF at this https URL.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2021 TransformerisallYouNeedMultimod	Amanpreet Singh Ronghang Hu			Transformer is all You Need: Multimodal Multitask Learning with a Unified Transformer						2021