Purpose: Technological evolution is radically changing medical learning models. We evaluated the learning outcomes of urological concepts using ChatGPT, traditional lecture and combined approach. Methods: We conducted a randomized triple-blind study on 121 medical students with no previous formal curriculum in urology. Students were randomly divided into three study classes with different learning methods: ChatGPT, Lecture and ChatGPT + Lecture. The "adrenal glands" were randomly extracted as the subject of the lessons. Students were evaluated using a thirty-question test. Results: The evaluation test median score was higher for students who underwent ChatGPT + Lecture compared with those who had only ChatGPT (10 vs. 12, p = 0.007). Such differences remained statistically significant also in multivariable models adjusting according to year of course, gender and previous ChatGPT experience (estimate: 2.6, p-value = 0.002). For most of the questions (about 70%), the proportion of students correctly answering was higher in the ChatGPT + Lecture learning groups than in the other groups. Conclusion: ChatGPT loses its potential if used without a previous background. The limits of scientific reliability persist and a teacher-guided method is still essential. ChatGPT + traditional lecture gives more effective results than the single traditional lecture also allowing a better use of the chatbot.

Chatgpt vs traditional pedagogy: a comparative study in urological learning

Digiacomo, Alessio
;
Orsini, Angelo;Cicchetti, Rossella;De Santis, Sara;Di Sessa, Laura;Di Nicola, Marta;Tamborino, Flavia;Basconi, Martina;De Archangelis, Riccardo;Salzano, Gaetano;Dello Stritto, Guglielmo;Lannutti, Peppino;Schips, Luigi;Marchioni, Michele
2025-01-01

Abstract

Purpose: Technological evolution is radically changing medical learning models. We evaluated the learning outcomes of urological concepts using ChatGPT, traditional lecture and combined approach. Methods: We conducted a randomized triple-blind study on 121 medical students with no previous formal curriculum in urology. Students were randomly divided into three study classes with different learning methods: ChatGPT, Lecture and ChatGPT + Lecture. The "adrenal glands" were randomly extracted as the subject of the lessons. Students were evaluated using a thirty-question test. Results: The evaluation test median score was higher for students who underwent ChatGPT + Lecture compared with those who had only ChatGPT (10 vs. 12, p = 0.007). Such differences remained statistically significant also in multivariable models adjusting according to year of course, gender and previous ChatGPT experience (estimate: 2.6, p-value = 0.002). For most of the questions (about 70%), the proportion of students correctly answering was higher in the ChatGPT + Lecture learning groups than in the other groups. Conclusion: ChatGPT loses its potential if used without a previous background. The limits of scientific reliability persist and a teacher-guided method is still essential. ChatGPT + traditional lecture gives more effective results than the single traditional lecture also allowing a better use of the chatbot.
File in questo prodotto:
File Dimensione Formato  
s00345-025-05654-w.pdf

accesso aperto

Tipologia: PDF editoriale
Dimensione 1.62 MB
Formato Adobe PDF
1.62 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11564/857133
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact