A mathematical model for assessing the applicability of intelligent chatbots for studying foreign language dialects
Abstract
A mathematical model for assessing the applicability of intelligent chatbots for studying foreign language dialects
Incoming article date: 25.11.2023The article presents a mathematical model for assessing the applicability of intelligent chatbots in the context of studying dialects of foreign languages. The model is based on the analysis of key parameters and characteristics of chatbots, as well as their ability to adapt to various dialects. The model's parameters include questions, answers, evaluation criteria, types, and costs of errors. The quality of the chatbot's responses is evaluated both according to individual criteria and overall. To test the effectiveness of the proposed method, an experimental study was conducted using the dialects of the German language as examples. During the research, such intelligent chatbots as ChatGPT-3.5, GPT-4, YouChat, Bard, DeepSeek, and Chatsonic were evaluated. The analysis of the results of applying the developed mathematical model showed that at present, the models by OpenAI (ChatGPT-3.5 and GPT-4) offer the broadest range of possibilities. ChatGPT-3.5 demonstrated the best results in communication in Bavarian and Austrian dialects, while YouChat excelled in the Swiss dialect. The obtained results allow for important practical recommendations to be made for selecting intelligent chatbots in the field of studying dialects of foreign languages and serve as a basis for further research in the area of evaluating the effectiveness of educational technologies based on artificial intelligence.
Keywords: large language model, chatbot, quality assessment, foreign language learning, artificial intelligence technology in education