Large language models (LLMs) are increasingly used as individual ideation assistants in project-based data science courses, yet little is known about whether students experience AI-generated ideas as shareable with peers. This paper presents a structural analysis of student evaluations of EduIDEAtor, a minimal LLM-based ideation tool for early project formulation. Using a pilot dataset of 16 post-use survey responses, we examine whether students' ratings of collaboration potential co-vary with assessments of other quality dimensions — engagement, creativity, ease of use, and project relevance, or whether collaboration potential functions as a distinct evaluative dimension. Descriptive analysis shows that 79% of respondents rated the tool as "Better" or "Much Better" than traditional brainstorming on collaboration potential (median = 4.0 on a 5-point scale). However, Spearman correlation analysis reveals that these ratings are decoupled from the usability and engagement cluster: the four other comparative dimensions form a tightly intercorrelated group (ρ = 0.54–0.80, all p < .05), while collaboration potential correlates significantly with none of them except project relevance (ρ = 0.65, p = .013). Cronbach's α for the five-item comparative block increases from 0.81 to 0.86 when collaboration potential is removed, providing psychometric confirmation that it does not track the same quality signal as the other indicators. These patterns suggest that students evaluate whether ideas feel shareable through a different interpretive lens than they use to assess usability or creativity. Improving interface design alone may not address this gap, therefore pedagogical structures that scaffold the social use of AI-generated ideas appear necessary.

Authors:
Stefania Zourlidou, University of Koblenz, Germany
Kamyab Farokhi, University of Koblenz, Germany
Frank Hopfgartner, University of Koblenz, Germany

About the Presenter(s)
Dr. Ing. Stefania Zourlidou is a postdoctoral researcher and academic advisor at the University of Koblenz, Germany.

See this presentation on the full schedule – On Demand Schedule

Bookmark