A nagy nyelvi modellek alkalmazhatóságának áttekintése a hatósági eljárások során

Róbert Karsa; Imre Négyesi

doi:10.61790/vt.2024.15993

Róbert Karsa
Imre Négyesi

DOI: https://doi.org/10.61790/vt.2024.15993

Keywords: artificial intellugence, natural language processing, large language models, public administration, principle of equivalence

Abstract

In recent years, research in artificial intelligence and machine learning has increasingly moved towards natural language processing (NLP), especially in the fields of large language models (LLM) and natural language understanding (NLU). The goal of NLP is to make computers capable of understanding different languages and processing textual information, which can bring about revolutionary changes similar to the spread of book printing or the Internet at the time. Significant progress has been made in the field of digitalization processes in public administration, with online forms of communication coming to the fore. Such innovative technologies as GPT-3 and its successors can improve official decision-making and communication through text understanding, machine translation and text classification. The aim of the research is to develop methods that can be used to make administrative official procedures and contact forms more efficient and transparent with the help of large language models. In the course of our research, we are undertaking the creation of a language model operating in a closed system, which can help the tasks of disaster protection authorities. We review the development of large language models, paying particular attention to the application possibilities of transformer-based models such as BERT and GPT in text comprehension and text generation. We present the processes of public administrative authority procedures and the points where machine learning methods can be effectively applied. During the research, we pay particular attention to examining the language models from a legal point of view, guaranteeing the preservation of legality and transparency. Based on the results, we make suggestions on how the large language models can be applied in the framework of official procedures, ensuring the increase of efficiency and transparency in public administration processes.

References

B. Hohmann, „Chatbotok a kormányzati platformok szolgálatában”, BELÜGYI SZEMLE: A BELÜGYMINISZTÉRIUM SZAKMAI TUDOMÁNYOS FOLYÓIRATA (2010-) 71 : 4, pp. 691-709., 2023.

D. Jurafsky és J. H. Martin, „Speech and Language Processing (3rd ed. draft)” 2023. [Online]. Elérhetőség: https://web.stanford.edu/~jurafsky/slp3/3.pdf (2023.11.01.)

A. Vaswani, „Attention Is All You Need,” Advances in Neural Information Processing Systems, pp. p./pp. 5998--6008, 2017.

T. Mikolov, „Efficient Estimation of Word Representations in Vector Space.,” 2013. [Online]. Elérhetőség: https://arxiv.org/pdf/1301.3781.

J. Chorowski, „Attention-based models for speech recognition.,” In Neural Information Processing Systems, p. pp. 577–585, 2015.

K. Hornik, M. Stinchcombe és W. Halbert, „Multilayer Feedforward Networks are Universal Approximators.,” Neural Networks. Vol. 2. Pergamon Press., p. pp. 359–366., 1989.

J. Devlin, „BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.,” arXiv preprint, p. arXiv:1810.04805, 2018.

D. Nemeskei, „Értsük meg a magyar entitásfelismerő rendszerek viselkedését!,” XVII. Magyar Számítógépes Nyelvészeti Konferencia,, p. pp. 409–418. , 2021.

Z. G. Yang, „Automatikus összefoglaló generálás magyar nyelvre BERT modellel.,” XVI. Magyar Számítógépes Nyelvészeti Konferencia,, p. pp. 319–329., 2020.

A. N. K. S. T. &. S. I. Radford, „Improving language understanding by generative pre-training.,” 2018.

A. Luccioni, „Estimating the carbon footprint of BLOOM, a 176B parameter language model.,” arXiv (Cornell University)., 2022.

Á. Feldmann, „HILBERT, magyar nyelvű BERT-large modell tanítása,” XVII. Magyar Számítógépes Nyelvészeti Konferencia, pp. pp. 29-36., 2021.

H. Touvron, „Llama 2: Open foundation and Fine-Tuned chat models,” arXiv.org, 2023b.

Z. G. a. D. Yang, „Jönnek a nagyok! BERT-Large, GPT-2 és GPT-3 nyelvmodellek magyar nyelvre,” XIX. Hungarian Computational Linguistics Conference, pp. 247--262, 2023.

Overview of the applicability of large language models during authority procedures

Abstract

References