ANALYSIS OF QUESTION MODIFICATION EFFECT BASED ON BLOOM'S TAXONOMY TOWARDS CONFIDENCE LEVEL OF CHATGPT USING GENETIC APPROACH METHODOLOGY

Abstract

In its mission to provide benefits to humanity through artificial intelligence, the OpenAI company continues to develop artificial intelligence technology innovations that focus on the language development model: ChatGPT. As of February 2023, OpenAI has more than 100 million users with an estimated 1.8 billion ChatGPT visits every month which beats TikTok and Instagram. This innovation helps users get answers based on the commands entered. The presence of technological innovation throughout history has had a positive impact on developments in various fields, including education. However, negative impacts also exist side by side and have become topics of discussion in various media. One study found that ChatGPT can help complete essay writing assignments in language subjects. This certainly has an impact on students losing opportunities to hone their cognitive abilities to create written works through language development exercises.

To reduce the negative impact of this technological innovation, it is necessary to carry out mitigation efforts which are expected to reduce students' dependence on using ChatGPT to complete their assignments at school. One effort that can be made is by modifying the question items by educators which can reduce ChatGPT's level of confidence in the answers given. To achieve this aim, this research took samples from the essay questions for the English class 12 high school exams. Through the Genetic Approach, each question item will be modified based on the selection of operational verbs from Bloom's Taxonomy until an operational verb (KKO) is found that is able to reduce ChatGPT's level of confidence in answering the question item. The Genetics Approach begins with the stages of population determination, namely the essay questions mentioned above, and chromosome determination, namely KKO at cognitive levels based on Bloom's Taxonomy. Next is the stage of determining the level of confidence through commands entered into ChatGPT after the answers to the modified question items are produced. If the level of confidence is still high, a crossover stage will be carried out by taking KKO at different cognitive levels until a lower level of confidence is obtained.

The results of the modification of the question items have shown a decrease in the average level of ChatGPT confidence in the answers produced from 0.91 to 0.65. The chromosomes (KKO) that have been tested can reduce the level of confidence, including generate, design, arrange, distinguish, organize, examine, formulate and sketch. Combining and crossing over chromosomes in each question item has a stronger effect on reducing the confidence level of ChatGPT compared to questions that only contain one chromosome in a question item.

Keywords:

ChatGPT, Bloom’s Taxonomy Keywords, Bloom’s Taxonomy, Genetic Approach