Development of RAG Chatbot for Academic and Government Information Services

Abstract

demand for information that is repetitive and often dispersed across official documents, web portals, circular letters, regulations, and standard operating procedures. Conventional rule-based chatbots are able to answer simple questions, but they are less adaptive to variations in natural language, policy changes, and the need for answers supported by source evidence. This article proposes a Retrieval-Augmented Generation (RAG)-based chatbot development framework to improve the accuracy, traceability, and security of information services in higher education institutions and government agencies. The proposed framework includes document acquisition, preprocessing, text segmentation, embedding generation, storage in a vector database, retrieval, reranking, answer generation by a large language model, source citation, security guardrails, and multilayered evaluation. The research method adopts a design science research approach through needs analysis, architecture design, RAG pipeline development, testing design, and risk mapping with mitigation strategies. The recommended evaluation metrics include precision@k, recall@k, mean reciprocal rank, context precision, context recall, faithfulness, answer relevancy, answer correctness, response latency, and the System Usability Scale. The design results indicate that a RAG chatbot has the potential to serve as a more accurate and auditable information service layer than a generic chatbot, particularly when the system is required to generate answers based on up-to-date official documents. The main contribution of this article is an architectural and evaluation framework that can serve as an initial blueprint for implementing RAG chatbots in academic services and digital public services.