Next Article in Journal
Wavelets and Real Interpolation of Besov Spaces
Next Article in Special Issue
Definition Extraction from Generic and Mathematical Domains with Deep Ensemble Learning
Previous Article in Journal
An Improved Genetic Algorithm for the Optimal Distribution of Fresh Products under Uncertain Demand
Article

To Batch or Not to Batch? Comparing Batching and Curriculum Learning Strategies across Tasks and Datasets

Computer Science and Engineering, University of Michigan, Ann Arbor, MI 48109, USA
*
Author to whom correspondence should be addressed.
Current address: 2260 Hayward Street, Ann Arbor, MI 48109, USA.
Academic Editors: Florentina Hristea, Cornelia Caragea and David Pugalee
Mathematics 2021, 9(18), 2234; https://doi.org/10.3390/math9182234
Received: 28 June 2021 / Revised: 3 September 2021 / Accepted: 4 September 2021 / Published: 11 September 2021
Many natural language processing architectures are greatly affected by seemingly small design decisions, such as batching and curriculum learning (how the training data are ordered during training). In order to better understand the impact of these decisions, we present a systematic analysis of different curriculum learning strategies and different batching strategies. We consider multiple datasets for three tasks: text classification, sentence and phrase similarity, and part-of-speech tagging. Our experiments demonstrate that certain curriculum learning and batching decisions do increase performance substantially for some tasks. View Full-Text
Keywords: natural language processing; word embeddings; batching; word2vec; curriculum learning; text classification; phrase similarity; part-of-speech tagging natural language processing; word embeddings; batching; word2vec; curriculum learning; text classification; phrase similarity; part-of-speech tagging
Show Figures

Figure 1

MDPI and ACS Style

Burdick, L.; Kummerfeld, J.K.; Mihalcea, R. To Batch or Not to Batch? Comparing Batching and Curriculum Learning Strategies across Tasks and Datasets. Mathematics 2021, 9, 2234. https://doi.org/10.3390/math9182234

AMA Style

Burdick L, Kummerfeld JK, Mihalcea R. To Batch or Not to Batch? Comparing Batching and Curriculum Learning Strategies across Tasks and Datasets. Mathematics. 2021; 9(18):2234. https://doi.org/10.3390/math9182234

Chicago/Turabian Style

Burdick, Laura, Jonathan K. Kummerfeld, and Rada Mihalcea. 2021. "To Batch or Not to Batch? Comparing Batching and Curriculum Learning Strategies across Tasks and Datasets" Mathematics 9, no. 18: 2234. https://doi.org/10.3390/math9182234

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop