Joseph A. Rios

Title

Translating/Adapting Tests to Increase Fairness for Linguistic Minority Examinees: Best Practice Recommendations for Test Development and Validity Efforts

October 1, 2019 (Tuesday), 11:00 a.m.-12:30 p.m. CDT

Abstract

This presentation will discuss the use of test adaptation as an accommodation for assessing linguistic minority populations. Although translating or adapting tests and questionnaires for use across multiple languages is not new, cross-lingual assessment activities are rapidly increasing. In fact, assessing individuals who operate in different languages through the use of test adaptation is one of the distinguishing features of assessment in the 21st century. However, there are serious concerns about the degree to which translated tests provide valid comparisons of examinees who take different language versions of an assessment.

To assist in improving the validity of inferences made from adapted tests, this presentation will discuss recommended test development efforts, such as identification of appropriate test translators and judgmental designs for evaluating translation quality. Furthermore, three potential sources of translation bias (construct, method, and item bias) will be discussed. In doing so, statistical methods for evaluating measurement invariance for each source of bias will be presented. When presenting statistical methods, which will include factor analysis, generalizability theory, and differential item functioning, a brief description of each procedure will be provided taking into consideration practical issues, such as small sample sizes, where possible. In addition to quantitative methodology, the presentation will also include a brief discussion on how qualitative methods (e.g., think-aloud protocol and cognitive interviewing) can be applied to concerns of test adaptation quality.

Bio

Joseph A. Rios is an assistant professor in the Quantitative Methods in Education program at the University of Minnesota. He holds a PhD in educational measurement and psychometrics from the University of Massachusetts, Amherst. Prior to joining the faculty at the University of Minnesota, Dr. Rios worked four years in the research and development division at ETS. He is currently a consulting editor of the Journal of Educational Psychology, and his work has been published in peer-reviewed journals, such as Educational Measurement: Issues and Practice, Applied Measurement in Education, International Journal of Testing, and Educational Assessment.

One of his primary research interests is in understanding how to improve the assessment of linguistic minorities. This interest stems from the increasing population of English language learners in the U.S. public education system as well as the desire by countries to compare international student achievement (e.g., PISA). To assist in confronting this issue, Dr. Rios' research focuses on validity concerns around testing accommodations, such as linguistic adaptation of test content.