Utomo Pujianto, Aldino Kelvin Nanda, Deni Sutaji
This study explores the feasibility of using conversation complexity between students and ChatGPT as a predictor of academic performance. Drawing on interaction logs from 76 undergraduate students in a Data Science course, five conversation attributes were extracted: average word count, prompt word count, number of prompts, discussion continuity score, and cognitive level based on Bloom's taxonomy. A neural network classifier was applied to predict academic performance labels, which were derived using unsupervised K-Means clustering on Mid-Semester Exam (UTS) scores. The model achieved an accuracy of 6 5. 7 9% and an F 1-score of 6 6. 7 5% for the 'poor' class and 64.86% for the 'good' class. While results indicate moderate predictive potential, limitations related to data validity, feature objectivity, and exam fairness warrant caution. This study contributes to the emerging field of AImediated learning analytics by offering a pilot exploration of ChatGPT conversation data as a digital learning trace. © 2025 IEEE.
Universitas Negeri Malang, Department of Electrical Engineering and Informatics, Malang, Indonesia; Gazi Üniversitesi Emniyet, Bilgisayar Bilimleri (Computer Science), Ankara, Turkey
