Transformer-Based Composite Language Models for Text Evaluation and Classification
Објеката
- Тип
- Рад у часопису
- Верзија рада
- објављена верзија
- Језик
- енглески
- Креатор
- Mihailo Škorić, Miloš Utvić, Ranka Stanković
- Извор
- Mathematics
- Издавач
- MDPI AG
- Датум издавања
- 2023
- Сажетак
- Parallel natural language processing systems were previously successfully tested on the tasks of part-of-speech tagging and authorship attribution through mini-language modeling, for which they achieved significantly better results than independent methods in the cases of seven European languages. The aim of this paper is to present the advantages of using composite language models in the processing and evaluation of texts written in arbitrary highly inflective and morphology-rich natural language, particularly Serbian. A perplexity-based dataset, the main asset for the methodology assessment, was created using a series of generative pre-trained transformers trained on different representations of the Serbian language corpus and a set of sentences classified into three groups (expert translations, corrupted translations, and machine translations). The paper describes a comparative analysis of calculated perplexities in order to measure the classification capability of different models on two binary classification tasks. In the course of the experiment, we tested three standalone language models (baseline) and two composite language models (which are based on perplexities outputted by all three standalone models). The presented results single out a complex stacked classifier using a multitude of features extracted from perplexity vectors as the optimal architecture of composite language models for both tasks.
- том
- 11
- Број
- 22
- doi
- 10.3390/math11224660
- issn
- 2227-7390
- Subject
- General Mathematics, Engineering (miscellaneous), Computer Science (miscellaneous)
- Шира категорија рада
- M20
- Ужа категорија рада
- М21а
- Права
- Отворени приступ
- Лиценца
- All rights reserved
- Формат
- Медија
mathematics-11-04660.pdf
Mihailo Škorić, Miloš Utvić, Ranka Stanković. "Transformer-Based Composite Language Models for Text Evaluation and Classification" in Mathematics, MDPI AG (2023). https://doi.org/10.3390/math11224660
This item was submitted on 17. новембар 2023. by [anonymous user] using the form “Рад у часопису” on the site “Радови”: http://gabp-dl.rgf.rs/s/repo
Click here to view the collected data.