BIO

I am a first-year computer science student from Miami, Florida. I am currently an undergraduate research assistant at the FSU Machine Learning Lab focusing on large language models. In my free time, I compete for the FSU Club Track and Field team in distance events and play the clarinet. After graduating, I hope to become a software engineer focusing on backend development.

Problem-Solving with an AI Assistant: A Usability Test

Abstract

Recent improvements in hardware, software, and cost-effective cloud services have allowed artificial intelligence (AI) technologies to be widely available on the consumer market. In the domain of higher education, current research focuses on administrative applications, with few focusing on how students can use AI to enhance their educational experience. This paper explores this knowledge gap and aims to help us understand how people interact with AI agents that can assist with learning. This research employed a usability test asking participants to solve a series of worded logic problems. During this test, undergraduate participants used Monica, a large language model interface to help them solve the problems. Participants were asked to describe their actions and thought processes while using the AI agent in a “think-aloud” protocol. The user interactions were transcribed and analyzed for keywords, themes, and topics that might best contextualize the interactions between the participants and the AI Agent.

Keywords: artificial intelligence, large language models, chatbots