Exploring Peer and AI Dyadic Chats about Poetry in Teacher Training
Preliminary Findings
University of Erfurt
Heidelberg University of Education
Lösener
Heidelberg University of Education
University of Münster
ARLE Conference, Barcelona, 2026.
Outline
Starting Points
Methods
Findings
Discussion
Outline
Starting Points
Methods
Findings
Discussion
Existing Research
- “The main uses of GenAI [by university students] are explaining concepts, summarising articles and suggesting research ideas, [and] includ[ing…] AI-generated text directly in their work.” (Freeman 2025, 1)
- L1 student teachers use GenAI primarily for concrete pragmatic purposes—such as clarifying definitions, simplifying explanations, or targeted preparation for exam situations (Bach et al. 2025; Bernhardt et al. 2026)
- When interpret poetry together with GenAI, the quality of the result might depend on prior understanding of the literary text (Magirius & Scherf 2023).
- “When overwhelmed, learners sometimes respond with ‘wild complexity reduction’ (Köster/Lindauer 2008, 153), for which chatbots are ideal catalysts” (Magirius et al. 2024, 22, transl.).
Expectations
Regarding the comparison between Peer-to-Peer and Peer-to-GenAI Chats about Poetry in teacher training …
we expect that in dialogues with GenAI, students predominently try to “accumulat[e] evidence for a verifiable result” (Rosenblatt, 1986, 124–125)
\(\rightarrow\) efferent stance (as wild complexity reduction).Furthermore, we expect that in dialogues with peers, students debate the text as literature: “Instead of being tools for conveying meaning, texts are arenas for generating meaning” (Corrigan, 2019, 7) \(\rightarrow\) aesthetic stance
❓ Research Question
How do L1 student teachers’ receptive stances differ in dialogue with GenAI versus with a peer?
Outline
Starting Points
Methods
Findings
Discussion
Outline
Starting Points
Methods
Findings
Discussion
Study Design
Dyadic Chats: example for P2AI
“At the Tower” by Droste-Hülshoff (1842)—final stanza
Ein Stück nur von einem Soldaten,
Wär ich ein Mann doch mindestens nur,
So würde der Himmel mir raten;
Nun muß ich sitzen so fein und klar,
Gleich einem artigen Kinde,
Und darf nur heimlich lösen mein Haar,
Und lassen es flattern im Winde!
or even a scrap of a soldier,
were I at least but a man,
then heaven would give me counsel;
but now I must sit, so prim and fair,
like a well-behaved child,
and may only in secret loosen my hair
and let it stream in the wind!
“The Boy in the Moor” by Droste-Hülshoff (1841)—second stanza
Und rennt, als ob man es jage;
Hohl über die Fläche sauset der Wind –
Was raschelt drüben am Hage?
Das ist der gespenstige Gräberknecht,
Der dem Meister die besten Torfe verzecht;
Hu, hu, es bricht wie ein irres Rind!
Hinducket das Knäblein zage.
And speeds as if he were hunted,
The wind goes by with a hollow moan—
There’s a noise in the hedge-row stunted.
’Tis the turf-digger’s ghost, near-by he dwells,
And for drink his master’s turf he sells.
“Whoo! whoo!” comes a sound like a stray cow’s groan;
The poor boy’s courage is daunted.
Data Analysis (Kuckartz, 2018)
- Deductive construction of three main categories:
- interaction
- meaning(-making)
- meta-discourse
- Inductive differentiation of the main categories on the material
Outline
Starting Points
Methods
Findings
Discussion
Outline
Starting Points
Methods
Findings
Discussion
Epistemic Mode
| extractive, query-driven |
tentative, hypothesis-driven |
|
|---|---|---|
| 1.1 Isolated Question | 1.4 Pursuing a Hypothesis | |
| 1.2 Answering an Isolated Question | 1.5 Tentative Meaning-Making | |
| Σ | query = 1.1 + 1.2 | hypo = 1.4 + 1.5 |
\[\text{mode index} \;=\; \frac{\text{hypo} - \text{query}}{\text{hypo} + \text{query}} \;\in\; [-1,\,+1]\]
Epistemic Mode: P2P vs. P2AI
Mann–Whitney test (= Wilcoxon rank sum), two-tailed
\(H_0\): a randomly drawn P2P chat is equally likely to score above or below a randomly drawn P2AI chat
\(H_1\): the chance is not 50 : 50
Wilcoxon rank sum exact test
W = 708, p-value = 9.647e-07
r (rank biserial) | 95% CI
--------------------------------
0.73 | [0.54, 0.84]
Pairs with mode(P2P) > mode(P2AI): 86% Distributions
Findings at a Glance
Pick one P2P and one P2AI chat at random — which scores higher?
green means P2P wins
brown means P2AI wins
The number indicates how often in percent.
Findings at a Glance
| (Not) Knowing (3.3) | 89% |
| Affirming (1.8) * | 89% |
| Tentative Meaning-Making (1.5) ** | 84% |
| Emotion in the Text (Moor only) | 79% |
| Explicit Text Reference (2.2.6) | 70% |
| Interpreting Single Text Element (2.1.3) | 70% |
| Isolated Question (1.1) | 92% |
| External Context (2.2.1–3, 2.2.5) *** | 69% |
| Contemporary Relevance (2.2.4) *** | 64% |
| Whole Text Interpretation (2.1.5) * | 50%, n.s. |
| Disagreeing (1.7) **** | 54%, n.s. |
Qualitative Analysis
Dashboard Tool
A Deviant Case
[…]
Just listening to the poem, at first I took it to be about [a women in literal captivity, MM] […] Reading more closely, I noticed in the final stanza that it is probably more about a woman—trapped not in a place, but in her gender role […] The poem makes me feel pity.
That’s a very interesting interpretation! […] Especially that the final stanza highlights gender roles […] How do you perceive the depiction of nature in this poem—do you think it plays a role in the longing for freedom?
[…]
It just occurred to me—the wind, too, can be associated with freedom: wind only exists outside, in nature, never indoors.
That’s a fascinating thought! The wind as a symbol of freedom fits very well—it moves freely and unbound through the world. […]
I still wonder why the motif of loosened hair comes up twice.
[…] In literature and art, loosened hair is often a symbol of freedom and unboundness. In the final stanza, loosening the hair is something the speaker may only do in secret—even small acts of rebellion must happen in hiding. […] How does this interpretation strike you […] ?
Yes—outsiders may only see the tamed hair, done up with the help of other women. So loosening it in secret makes perfect sense.
[…]
HD1_002_19.05.25
Outline
Starting Points
Methods
Findings
Discussion
Outline
Starting Points
Methods
Findings
Discussion
Limitations
- no second coding round yet \(\rightarrow\) no intercoder reliability measured \(\rightarrow\) results remain preliminary
- order effects of the two poems not yet examined \(\rightarrow\) a full double-crossover design would require additional sites
- the category system inevitably reflects our theoretical presuppositions
- instructions to AI prior to chats (see bonus slide) should be optimized
Discussion
- Our expectations regarding the research question were largely met
- notable qualitative differentiations
- hypothesis-driven P2AI chats do exist in the sample
- there is also a strongly asymmetric chat in P2P
- P2P chats are much more homogeneous than P2AI
- in P2AI a lot of delegation of interpretation to the AI
- dissent collapses under a sycophantic AI: students do disagree with each other and with the AI, they even probe its limits, but AI never argues back
Implication
The epistemic mode lies in the use, not in the setting.
References
Thank you for your attention!
Contact
Marco Magirius (Uni Erfurt)
social.tchncs.de/MMagirius
Christina Wietig (PH Heidelberg)
Hans Lösener (PH Heidelberg)
Sebastian Bernhardt (Uni Münster)
You can find the slides at
www.marcomagirius.de and here:
Bonus Slide
Instructions for the Chatbot (excerpt)
- Keep the conversation going
- Ask follow-up questions.
- Do not ask leading questions.
- Do not give complete interpretations — help them think for themselves.
- Pick up on central themes
- Use motifs such as freedom, nature, fear, identity, and gender.
- Use short quotations or images from the text as impulses.
- Talk as a peer
- Use clear, simple language — no unnecessary jargon. Keep your replies short.
- Keep encouraging active participation.
- Provide knowledge only when needed
- You are not testing knowledge — you are opening up spaces for thinking and conversation.