IMPLEMENTASI QUESTION ANSWERING SYSTEM TAFSIR AL-AZHAR MENGGUNAKAN  LANGCHAIN DAN LARGE LANGUAGE MODEL BERBASIS CHATBOT TELEGRAM

Aji Bayu Permadi; Nazruddin Safaat H; Lestari Handayani; Yusra

doi:10.21063/jtif.2024.V12.1.62-69

Authors

Aji Bayu Permadi
Nazruddin Safaat H
nazruddin.safaat@uin-suska.ac.id
Universitas Islam Negeri Sultan Syarif Kasim Riau
Lestari Handayani
Yusra

Keywords:

Question Answering System, Langchain, Large Language Model , Tafsir Al-Quran , Chatbot

Abstract

Tafsir is a main gateway for a Muslim to study and understand the content of the verses in the Quran. One example is Tafsir Al-Azhar. Tafsir Al-Azhar is a commentary authored by Professor Dr. Hamka, which demonstrates how Dr. Hamka connects modern Islamic history with Quranic studies. Tafsir Al-Azhar has a large number of pages, requiring extra effort when searching for information within it. This research aims to create a system capable of receiving questions about Tafsir Al-Azhar in natural language and answering them in user-friendly terms. The technology used in this research includes Langchain and Large Language Models, implemented using a Telegram chatbot. Telegram was chosen for its popularity and user-friendly interface. The Question Answering system was tested using User Acceptance Testing (UAT) and the DeepEval framework. The UAT resulted in an accuracy score of 83.71%, while testing using the DeepEval framework yielded a hallucination score of 41%, contextual precision of 90%, contextual recall of 81%, and contextual relevancy of 79%.

References

R. J. Sahputra and A. Muzakir, “Penerapan AI Melalui Pendekatan Heuristik Semilaritas Pada Game Edukasi Anak Usia Dini,” J. Pengemb. Sist. Inf. dan Inform., vol. 1, no. 4, pp. 209–219, 2021, doi: 10.47747/jpsii.v1i4.547.

T. Wahyudi, “Studi Kasus Pengembangan dan Penggunaan Artificial Intelligence (AI) Sebagai Penunjang Kegiatan Masyarakat Indonesia,” Indones. J. Softw. Eng., vol. 9, no. 1, pp. 28–32, 2023, [Online]. Available: https://ejournal.bsi.ac.id/ejurnal/index.php/ijse

A. Latif, “Spektrum Historis Tafsir Al-Qur’an Di Indonesia,” TAJDID J. Ilmu Ushuluddin, vol. 18, no. 1, pp. 105–124, 2020, doi: 10.30631/tjd.v18i1.97.

O. Yulianda, “Istidraj Menurut Hamka dalam tafsir Al-Azhar,” vol. 10, p. 6, 2021.

A. Dhandapani and V. Vadivel, “Question Answering System over Semantic Web,” IEEE Access, vol. 9, pp. 46900–46910, 2021, doi: 10.1109/ACCESS.2021.3067942.

R. A. Yunmar and I. W. W. Wisesa, “Pengembangan Mobile-Based Question Answering System Mobile-Based Question Answering System Development With Ontology Based Knowledge,” J. Teknol. Inf. dan Ilmu Komput., vol. 7, no. 4, pp. 693–700, 2020, doi: 10.25126/jtiik.202072255.

O. Topsakal and T. C. Akinci, “Creating Large Language Model Applications Utilizing LangChain: A Primer on Developing LLM Apps Fast,” Int. Conf. Appl. Eng. Nat. Sci., vol. 1, no. 1, pp. 1050–1056, 2023, doi: 10.59287/icaens.1127.

B. D. Laraswati, “Large Language Models, Cikal Bakal Lahirnya Chat GPT.” Accessed: Nov. 08, 2023. [Online]. Available: https://blog.algorit.ma/large-language-models/

A. Hariansyah, E. Haerani, and M. Affandes, “Implementation of Telegram Chatbot as Information Service of Madani Hospital Pekanbaru,” vol. 11, no. 3, pp. 188–198, 2023.

“Telegram Messenger - Tech Stack, Apps, Patents & Trademarks.” Accessed: Feb. 09, 2024. [Online]. Available: https://www.crunchbase.com/organization/telegram-messenger/technology

Heri Khariono, Rizky Parlika, Haidar Ananta Kusuma, and Dimas Arif Setyawan, “Pemanfaatan Bot Telegram Sebagai E-Learning Ujian Berbasis File,” J. Inform. Polinema, vol. 7, no. 4, pp. 65–72, 2021, doi: 10.33795/jip.v7i4.696.

M. A. P. Subali and P. Wijaya, “Sistem Question Answering untuk Bahasa Bali menggunakan Metode Rule-Based dan String Similarity,” Techno.Com, vol. 20, no. 2, pp. 300–308, 2021, doi: 10.33633/tc.v20i2.4390.

J. A. Alzubi, R. Jain, A. Singh, P. Parwekar, and M. Gupta, “COBERT: COVID-19 Question Answering System Using BERT,” Arab. J. Sci. Eng., vol. 48, no. 8, pp. 11003–11013, 2023, doi: 10.1007/s13369-021-05810-5.

P. Pujianto, M. Mujito, D. Prabowo, and B. H. Prasetyo, “Pemilihan Warga Penerima Bantuan Program Keluarga Harapan (PKH) Menggunakan Metode Simple Additive Weighting (SAW) dan User Acceptance Testing (UAT),” J. Inform. Univ. Pamulang, vol. 5, no. 3, p. 379, 2020, doi: 10.32493/informatika.v5i3.6671.

“DeepEval - The Open-Source LLM Evaluation Framework.” Accessed: Mar. 12, 2024. [Online]. Available: https://docs.confident-ai.com/

“Text Splitters | Langchain.” Accessed: Feb. 26, 2024. [Online]. Available: https://python.langchain.com/docs/modules/data_connection/document_transformers/

K. Pandya and B. V. Mahavidyalaya, “Automating Customer Service using LangChain,” Comput. Lang. (cs.CL); Comput. Soc. (cs.CY); Mach. Learn., vol. 1, pp. 28–31, 2023, [Online]. Available: https://arxiv.org/abs/2310.05421

M. Douze et al., “The Faiss library,” 2024, [Online]. Available: http://arxiv.org/abs/2401.08281

Graciela Fausten Novindri and P. Ocsa Nugraha Saian, “Implementasi Flask Pada Sistem Penentuan Minimal Order Untuk Tiap Item Barang Di Distribution Center Pada Pt Xyz Berbasis Website,” J. Mnemon., vol. 5, no. 2, pp. 81–85, 2022, doi: 10.36040/mnemonic.v5i2.4670.

B. P. Putra and Y. A. Susetyo, “Implementasi Api Master Store Menggunakan Flask, Rest Dan Orm Di Pt Xyz,” Sistemasi, vol. 9, no. 3, p. 543, 2020, doi: 10.32520/stmsi.v9i3.899.

M. Schwarz, K. Chapman, and B. Häussler, “Multilingual Medical Entity Recognition and Cross-lingual Zero-Shot Linking with Facebook AI Similarity Search,” CEUR Workshop Proc., vol. 3202, 2022.