Generative AI chatbot for teachers' data-informed decision-making: Effects and insights

Abstract

Teachers' instructional decisions rely on effective data collection, analysis, and interpretation. Generative artificial intelligence (AI) offers flexible and efficient support for these processes. However, few studies have explored its practical application in classroom decision-making. This study aimed to develop a generative AI chatbot to enhance teachers' data literacy (DL) and data-informed decision-making (DIDM). Using Richey and Klein's design and development research methodology, this study was conducted in four phases: analysis, design, development, and evaluation. The chatbot was reviewed by five experts and pilot-tested by four participants before being tested in an experiment. A single group pre-post paired samples t-test was conducted with 25 teachers, who interacted with the chatbot on Zoom once a week for 2 weeks. Teachers' reflective journals were examined using open coding procedures to analyze their responses. The findings revealed a significant increase in teachers' DL and DIDM efficacy. A qualitative analysis of the teachers' reflective journals highlighted the chatbot's strengths, limitations, and potential improvements. Based on these findings, this study offers practical implications for developing and using generative AI chatbots to support school teachers' DIDM processes.