Compositional-structural, semantic and presuppositional-pragmatic parameters and defects of generated short texts in the GigaChat language neural network

Introduction. The relevance of the study lies primarily in the fact that the increasingly active appeal of the widest circles of users to the generation of texts of different genres, properties and volumes using the so-called LLM (Large Language Model) creates the need to study the actual linguistic capabilities of these generative models, including the need for linguistic research of the texts they generate. It is also important that a large number of specialists from different fields actively use the generative capabilities of linguistic neural networks for purely professional purposes, which means that the quality of generated texts has acquired the status of a production factor and directly affects success in professional activities, which also indicates the relevance of our research.

Materials and Methods. The texts of short stories were chosen as the object of study, since in a certain respect, such texts can be considered as archetypal structures that underlie texts of certain functional types (scripts, advertising texts and stories).

Analysis. This state of affairs requires studying the generated stories, including in the aspect of the representation of frame structures in it, since this will allow us to get an idea of how texts of this type are structured from the point of view of standard linguistic semantics, including syntactic semantics.

Results. It has been established that, despite all grammatical, structural-semantic and compositional adequacy, the texts of short stories generated in GigaChatPro, created by the network on a certain topic or according to a certain frame, may contain violations in global semantic organization, generated by both presuppositional-pragmatic violations and violations in text reference. Violations in the general functional pragmatics of the generated stories were also noted: pronounced edification and the length of sentences up to 5-7 words are characteristic of stories for children of primary school age, despite the fact that the themes of these stories initially do not correspond to their age interests.

1. Arutyunova, ND. The sentence and its meaning: logical-semantic problems: monograph. Moscow: Nauka; 1976. 383 p. (In Russ.).

2. Gusarenko SV, Gusarenko MK. Predicate-frame description of the linear semantics of artistic narrative. Filologicheskie nauki. 2020;3:18-30. (In Russ.)

3. Deik TA van, Kinch V. Strategies for understanding a coherent text. Novoe v zarubezhnoi lingvistike: Kognitivnye aspekty yazyka. 1988;XXIII:153-211. (In Russ.).

4. Demchenkov SA, Fedyaeva ND. Verbal happening: about some models of computer generation of texts in the aspect of avantgarde poetics. Humanitarian Vector. 2020;15;1:27-37. (In Russ.).

5. Il'f I, Petrov E. The twelve Chairs. The Golden Calf: Novels. Notebooks. Moscow: Eksmo; 2007. 640 p. (In Russ.).

6. Kozlovskii AV, Mel'nik YaE, Voloshchuk VI. About the approach for automatic generation of plot-related text). Izvestiya TulGU. 2022;9:160-167. (In Russ.).

7. Krivosheev NA, Ivanova YuA, Spitsyn VG. Automatic generation of short texts based on the use of LSTM and SecGAN neural networks. Vestnik Tomskogo gosudarstvennogo universiteta. 2021;57:118-130. (In Russ.).

8. Kuchina SA. Electronic generative literary texts. Vestnik TGPU (TSPU Bulletin). 2017;6(183):49-55. (In Russ.).

9. Propp VYa. Morphology of a fairy tale. Moscow: Labirint; 2001. 192 p. (In Russ.).

10. GigaChat – Russian-language neural network from Sber. URL: https://developers.sber.ru/gigachat/login (accessed: 26.02.2024). (In Russ.).

11. Stolneiker R. Pragmatics. Novoe v zarubezhnoi lingvistike: Lingvisticheskaya pragmatika. 1985;XVI:419-438. (In Russ.)

12. Tokareva IA. Structural-semantic model of semantic-syntactic frame. Kazanskii lingvisticheskii zhurnal. 2022;5;4:547-558. (In Russ.).

13. GigaChatPro. URL: https://web.telegram.org/a/#6218783903 (accessed: 15.12.2023).

14. Goloviznina V, Fishcheva I, Peskisheva T, Kotelnikov E. Aspect-based Argument Generation in Russian in Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue”. (2023). Issue 22. URL: https://www.dialog-21.ru/digest/2023/articles/ (accessed: 15.02.2024).

15. Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang Y, Chen D, Chan HS, Dai W, Madotto A, Fung P. Survey of hallucination in natural language generation in ACM Computing Surveys. Vol. 1. No. 1. Article. URL: https://www. semanticschol-ar.org/search?q=Survey%20of%20hallucination%20in%20natural%20language%20generation&sort=relevance (accessed: 15.02.2024).

16. Maynez J, Narayan S, Bohnet B, Mcdonald RT. On Faithfulness and Factuality in Abstractive Summarization in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. P. 1906-1919.

17. Mikhaylovskiy N, Churilov I. Autocorrelations Decay in Texts and Applicability Limits of Language Models in Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue”. (2023). Issue 22. URL: https://www.dialog-21.ru/digest/2023/articles/ (accessed: 15.02.2024).

18. Ostyakova L, Petukhova K, Smilga V, Zharikova D. Linguistic Annotation Generation with ChatGPT: a Synthetic Dataset of Speech Functions for Discourse Annotation of Casual Conversations in Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue” (2023). Issue 22. URL: https://www.dialog-21.ru/digest/2023/articles/ (accessed: 15.02.2024).

19. Vychegzhanin S, Kotelnikova A, Sergeev A, Kotelnikov E. MaxProb: Controllable Story Generation from Storyline in Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue” (2023). Issue 22. URL: https://www.dialog-21.ru/digest/2023/articles/ (accessed: 15.02.2024).