Implementation of a chatbot based on artificial intelligence language model to respond to frequently asked questions from university students
DOI:
https://doi.org/10.51252/rcsi.v3i2.570Keywords:
conversational agent, virtual assistant, university care, GPT, student supportAbstract
The use of virtual assistants is common in the service processes of different universities around the world. Nowadays, several artificial intelligence tools such as natural language processors are offered for a more efficient development of this service. This article presents the implementation and usability evaluation of a chatbot based on the GPT-3.5-Turbo artificial intelligence language model for the attention of first semester students of Systems Engineering at the National University of Trujillo. The research design was pre-experimental and the chatbot usability questionnaire or CUQ was applied as a research instrument. The results obtained were that the chatbot presents an acceptable level of perceived usability and 93% of students agreed that the answers provided by the virtual assistant were useful. The conversational agent stood out in ease of use and navigation, and also exhibited problems in understanding the questions asked by the user and providing answers in a natural way.
Downloads
References
Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An Empirical Evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 24(6), 574–594. https://doi.org/10.1080/10447310802205776
Barus, S. P., & Surijati, E. (2022). Chatbot with Dialogflow for FAQ Services in Matana University Library. International Journal of Informatics and Computation, 3(2), 62. https://doi.org/10.35842/ijicom.v3i2.43
Busch, K., Rochlitzer, A., & Diana Sola, H. L. (2023). Just Tell Me: Prompt Engineering in Business Process Management. Computer Science, 1. https://arxiv.org/abs/2304.07183
Casazola Cruz, O. D., Alfaro Mariño, G., Burgos Tejada, J., & Ramos More, O. A. (2021). La usabilidad percibida de los chatbots sobre la atención al cliente en las organizaciones: una revisión de la literatura. Interfases, 014, 184–204. https://doi.org/10.26439/interfases2021.n014.5401
Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023). GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models. http://arxiv.org/abs/2303.10130
Holmes, S., Moorhead, A., Bond, R., Zheng, H., Coates, V., & Mctear, M. (2019). Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces? Proceedings of the 31st European Conference on Cognitive Ergonomics, 207–214. https://doi.org/10.1145/3335082.3335094
Hsu, C.-L., & Lin, J. C.-C. (2023). Understanding the user satisfaction and loyalty of customer service chatbots. Journal of Retailing and Consumer Services, 71, 103211. https://doi.org/10.1016/j.jretconser.2022.103211
Jimenez Flores, V. J. (2019). Entidad conversacional de inteligencia artificial y calidad del servicio percibido por estudiantes de la Universidad José Carlos Mariátegui Filial Tacna, 2018 – II. [Universidad Nacional Jorge Basadre Grohmann]. http://repositorio.unjbg.edu.pe/handle/UNJBG/3848
Jordan, P. W., Thomas, B., McClelland, I. L., & Weerdmeester, B. (1996). Usability Evaluation In Industry (1st ed.). CRC Press.
Kumar, H., Musabirov, I., Shi, J., Lauzon, A., Choy, K. K., Gross, O., Kulzhabayeva, D., & Williams, J. J. (2022). Exploring The Design of Prompts For Applying GPT-3 based Chatbots: A Mental Wellbeing Case Study on Mechanical Turk. http://arxiv.org/abs/2209.11344
Kumar, M. (2019). Serverless Architectures Review, Future Trend and the Solutions to Open Problems. American Journal of Software Engineering, 6(1), 1–10. https://doi.org/10.12691/ajse-6-1-1
Lee, K., Jo, J., Kim, J., & Kang, Y. (2019). Can Chatbots Help Reduce the Workload of Administrative Officers? - Implementing and Deploying FAQ Chatbot Service in a University (pp. 348–354). https://doi.org/10.1007/978-3-030-23522-2_45
Lewis, J. R., & Sauro, J. (2018). Item Benchmarks for the System Usability Scale. Journal of Usability Studies, 13(3), 158–167. https://uxpajournal.org/wp-content/uploads/sites/7/pdf/JUS_Lewis_May2018.pdf
Lin, Z., Xu, P., Winata, G. I., Siddique, F. Bin, Liu, Z., Shin, J., & Fung, P. (2020). CAiRE: An End-to-End Empathetic Chatbot. Proceedings of the AAAI Conference on Artificial Intelligence, 34(09), 13622–13623. https://doi.org/10.1609/aaai.v34i09.7098
Madhan Kumar, C., Fardeen, M., Rohit, P., & Rakhes, M. (2023). AI based chatbot to answer faqs. Interantional Journal of Scientific Research in Engineering and Management, 7(3). https://doi.org/10.55041/IJSREM18072
Mariani, M. M., Hashemi, N., & Wirtz, J. (2023). Artificial intelligence empowered conversational agents: A systematic literature review and research agenda. Journal of Business Research, 161, 113838. https://doi.org/10.1016/j.jbusres.2023.113838
Mohammed, C. M., & Zeebaree, S. R. . (2021). Sufficient Comparison Among Cloud Computing Services: IaaS, PaaS, and SaaS: A Review. International Journal of Science and Business, 5(2), 17–30. https://ideas.repec.org/a/aif/journl/v5y2021i2p17-30.html
Nugraha, K. A., & Sebastian, D. (2021). Proceedings of the 7th International Conference on Education and Technology (ICET 2021). Proceedings of the 7th International Conference on Education and Technology, 601, 119–122. https://www.atlantis-press.com/proceedings/icet-21/125964506
Ray, P. P. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems, 3, 121–154. https://doi.org/10.1016/j.iotcps.2023.04.003
Reddy K, G. K., Pal, A., V, S. K., J, R., & K, S. (2022). Cross Domain Answering FAQ Chatbot. 2022 International Conference on Advanced Computing Technologies and Applications (ICACTA), 1–4. https://doi.org/10.1109/ICACTA54488.2022.9752986
Sánchez, W. O. (2011). Logotipo del repositorio Comunidades Iniciar sesión Inicio Revistas y libros Revista Ing-Novación No. 02, agosto 2011 La usabilidad en Ingeniería de software: definición y características La usabilidad en Ingeniería de software: definición y característic. Revista Ing-Novación, 2, 7–22. http://hdl.handle.net/11715/519
Santana, R., Ferreira, S., Rolim, V., Miranda, P., Nascimento, A., & Mello, R. F. (2021). A Chatbot to Support Basic Students Questions. CEUR Workshop Proceedings, 3059, 58–67. http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-3059/paper6.pdf
Saranya, A., & Subhashini, R. (2023). A systematic review of Explainable Artificial Intelligence models and applications: Recent developments and future trends. Decision Analytics Journal, 7, 100230. https://doi.org/10.1016/j.dajour.2023.100230
Shaji George, A., & Sagayarajan, S. (2023). Securing Cloud Application Infrastructure : Understanding the Penetration Testing Challenges of IaaS , PaaS , and SaaS Environments Partners Universal International Research Journal ( PUIRJ ). Partners Universal International Research Journal, March, 24–34. https://doi.org/10.5281/zenodo.7723187
Sotnik, S., Shakurova, T., & Lyashenko, V. (2023). Development Features Web-Applications. International Journal of Academic and Applied Research, 7(1), 79–85. https://openarchive.nure.ua/handle/document/21600
Tamkin, A., Brundage, M., Clark, J., & Ganguli, D. (2021). Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models. http://arxiv.org/abs/2102.02503
Wang, X., Lin, X., & Shao, B. (2022). How does artificial intelligence create business agility? Evidence from chatbots. International Journal of Information Management, 66, 102535. https://doi.org/10.1016/j.ijinfomgt.2022.102535
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. http://arxiv.org/abs/2302.11382
Wolde, B. G., & Boltana, A. S. (2021). REST API Composition for Effective Testing the Cloud. Journal of Applied Research and Technology, 19(6), 676–693. https://doi.org/10.22201/icat.24486736e.2021.19.6.924
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Bruno Hiroshi Espinosa-Luna, Johann Castillo-Oliva, Bruno Adrián Montañez-Díaz, Alberto Mendoza-De-los-Santos
This work is licensed under a Creative Commons Attribution 4.0 International License.
The authors retain their rights:
a. The authors retain their trademark and patent rights, as well as any process or procedure described in the article.
b. The authors retain the right to share, copy, distribute, execute and publicly communicate the article published in the Revista Científica de Sistemas e Informática (RCSI) (for example, place it in an institutional repository or publish it in a book), with an acknowledgment of its initial publication in the RCSI.
c. Authors retain the right to make a subsequent publication of their work, to use the article or any part of it (for example: a compilation of their works, notes for conferences, thesis, or for a book), provided that they indicate the source of publication (authors of the work, journal, volume, number and date).
