In the educational field, it is common to analyze test data through Item Response Theory (IRT) models. In this context, a key role is played by item characteristic curves (ICCs) and item information curves (IICs). In many real cases, practitioners are interested in understanding if some factors have a significant influence on the probability of correctly answering items. In the literature, this problem has been addressed by applying the standard analysis of variance (ANOVA) model, which is based on the total scores or the proportion of correct responses. However, this method needs to meet some strong assumptions and may present some limitations because it does not consider useful information typical of the IRT, such as the shapes of the ICCs and IICs, which provide interesting insights for different ability levels. To overcome these issues, this research suggests the use of the functional analysis of variance (FANOVA) approach and a novel functional tool in the IRT context. The main advantages of this approach are that it is distribution-free and allows us to check the degree of consistency with the hypothesis of equality among mean curves for different ability levels. Specifically, the proposed method is applied on ICCs and IICs for improving the existing techniques in the educational studies. A real dataset drawn from the IPRASE Trentino Language Testing (TLT) Survey 2016 is considered. The final purpose of this study is to provide additional tools for scholars and practitioners in defining specific educational plans.

Testing equality of functions across multiple experimental conditions for different ability levels in the IRT context: the case of the IPRASE TLT 2016 Survey

Fabrizio Maturo
;
Francesca Fortuna;Tonio Di Battista
2018

Abstract

In the educational field, it is common to analyze test data through Item Response Theory (IRT) models. In this context, a key role is played by item characteristic curves (ICCs) and item information curves (IICs). In many real cases, practitioners are interested in understanding if some factors have a significant influence on the probability of correctly answering items. In the literature, this problem has been addressed by applying the standard analysis of variance (ANOVA) model, which is based on the total scores or the proportion of correct responses. However, this method needs to meet some strong assumptions and may present some limitations because it does not consider useful information typical of the IRT, such as the shapes of the ICCs and IICs, which provide interesting insights for different ability levels. To overcome these issues, this research suggests the use of the functional analysis of variance (FANOVA) approach and a novel functional tool in the IRT context. The main advantages of this approach are that it is distribution-free and allows us to check the degree of consistency with the hypothesis of equality among mean curves for different ability levels. Specifically, the proposed method is applied on ICCs and IICs for improving the existing techniques in the educational studies. A real dataset drawn from the IPRASE Trentino Language Testing (TLT) Survey 2016 is considered. The final purpose of this study is to provide additional tools for scholars and practitioners in defining specific educational plans.
File in questo prodotto:
File Dimensione Formato  
SIR-IES.pdf

Solo gestori archivio

Tipologia: Documento in Pre-print
Dimensione 3.86 MB
Formato Adobe PDF
3.86 MB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11564/688491
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 11
  • ???jsp.display-item.citation.isi??? 6
social impact