Adapters in Transformers. A New Paradigm for Transfer Learning? [ LingMon #182 ]
📅 November 29, 2021 | 🗣 Jonas Pfeiffer (Technical University of Darmstadt) | 📝 Adapters in Transformers: A New Paradigm for Transfer Learning?
🔖 ABOUT THE LECTURE
▔▔▔▔▔▔▔▔▔▔▔▔▔
▸Adapters have recently been introduced as an alternative transfer learning strategy. Instead of fine-tuning all weights of a pre-trained transformer-based model, small neural network components are introduced at every layer. While the pre-trained parameters are frozen, only the newly introduced adapter weights are fine-tuned, achieving an encapsulation of the down-stream task information in designated parts of the model. In this talk we will provide an introduction to adapter-training in natural language processing. We will go into detail on how the encapsulated knowledge can be leveraged for compositional transfer learning, as well as cross-lingual transfer. We will further briefly touch the efficiency of adapters in terms of trainable parameters as well as (wall-clock) training time. Finally, we will provide an outlook to recent alternative adapter approaches and training strategies.
📽️ ABOUT THE SERIES
▔▔▔▔▔▔▔▔▔▔▔▔
▸The history of Linguistic Mondays dates back to the 1980s and their original aim was to make both the students and the faculty members as well as the wider research community aware of the field of computational linguistics in general and of the results achieved by the members of our team in particular. During the years, with the growing awareness of the domain and with new trends appearing on the scene and with more master and doctoral students coming in, the scope of the topics introduced has broadened correspondingly, covering all aspects of the field from the basics of computational linguistics, its linguistic and formal background through corpora case studies and natural language processing applications such as machine translation and information retrieval up to the most modern trends including machine learning. It is also offers an excellent opportunity for PhD students to present their results and to receive a relevant response from leading experts in the field.
❔ SOCIAL MEDIA
▔▔▔▔▔▔▔▔▔▔
▸web: https://ufal.mff.cuni.cz
▸facebook: Facebook: UFALMFFUK
▸twitter: Twitter: ufal_cuni
🔖 ABOUT THE LECTURE
▔▔▔▔▔▔▔▔▔▔▔▔▔
▸Adapters have recently been introduced as an alternative transfer learning strategy. Instead of fine-tuning all weights of a pre-trained transformer-based model, small neural network components are introduced at every layer. While the pre-trained parameters are frozen, only the newly introduced adapter weights are fine-tuned, achieving an encapsulation of the down-stream task information in designated parts of the model. In this talk we will provide an introduction to adapter-training in natural language processing. We will go into detail on how the encapsulated knowledge can be leveraged for compositional transfer learning, as well as cross-lingual transfer. We will further briefly touch the efficiency of adapters in terms of trainable parameters as well as (wall-clock) training time. Finally, we will provide an outlook to recent alternative adapter approaches and training strategies.
📽️ ABOUT THE SERIES
▔▔▔▔▔▔▔▔▔▔▔▔
▸The history of Linguistic Mondays dates back to the 1980s and their original aim was to make both the students and the faculty members as well as the wider research community aware of the field of computational linguistics in general and of the results achieved by the members of our team in particular. During the years, with the growing awareness of the domain and with new trends appearing on the scene and with more master and doctoral students coming in, the scope of the topics introduced has broadened correspondingly, covering all aspects of the field from the basics of computational linguistics, its linguistic and formal background through corpora case studies and natural language processing applications such as machine translation and information retrieval up to the most modern trends including machine learning. It is also offers an excellent opportunity for PhD students to present their results and to receive a relevant response from leading experts in the field.
❔ SOCIAL MEDIA
▔▔▔▔▔▔▔▔▔▔
▸web: https://ufal.mff.cuni.cz
▸facebook: Facebook: UFALMFFUK
▸twitter: Twitter: ufal_cuni
همه توضیحات ...