AI Persona-Based Student Counseling Chatbot Using Large Language Model, RAG, and Prompt Engineering

Abstract

Chatbots are increasingly used in student counseling services because they offer easy access, fast responses, and flexible availability. However, conventional chatbots often produce generic responses, have limited contextual understanding, and provide insufficient emotional support. This study aims to develop an AI Persona-based student counseling chatbot using a Large Language Model (LLM), Retrieval-Augmented Generation (RAG), and prompt engineering to generate relevant, contextual, and empathetic responses. The study uses a Research and Development (R&D) approach with the CRISP-DM framework. The system uses Gemini 2.5 Flash as the generative model, multilingual-e5-small as the embedding model, and FAISS as the vector index. Four institutional documents and campus service data are processed through chunking, embedding, and semantic retrieval. Evaluation is conducted using LLM-as-a-Judge on 45 scenarios and User Acceptance Testing (UAT) with 20 students. The LLM-as-a-Judge evaluation produces an average score of 4.47 out of 5, with the highest score in Context Relevance at 4.70. UAT achieves 91% user acceptance in the very good category, with naturalness and empathy as the highest indicator at 95%. The results show that integrating LLM, RAG, and prompt engineering can improve chatbot response quality without fine-tuning, although further development is needed in multimodal document support, local model deployment, and retrieval mechanism improvement.