Translating/Adapting Tests to Increase Fairness for Linguistic Minority Examinees: Best Practice Recommendations for Test Development and Validity Efforts
October 1, 2019 (Tuesday), 11:00 a.m.-12:30 p.m. CDT
LEARNING OBJECTIVES
- Become familiar with the challenges of assessing linguistic minorities.
- Appreciate why test adaptation has been suggested to be one of the most effective test accommodations for assessing linguistic minorities.
- Develop an understanding of effective practices for adapting assessments.
- Appreciate validity concerns related to test adaptation.
- Understand measurement invariance approaches to investigate equivalence between source and targeted language versions.
Abstract
This presentation will discuss the use of test adaptation as an accommodation for assessing linguistic minority populations. Although translating or adapting tests and questionnaires for use across multiple languages is not new, cross-lingual assessment activities are rapidly increasing. In fact, assessing individuals who operate in different languages through the use of test adaptation is one of the distinguishing features of assessment in the 21st century. However, there are serious concerns about the degree to which translated tests provide valid comparisons of examinees who take different language versions of an assessment.
To assist in improving the validity of inferences made from adapted tests, this presentation will discuss recommended test development efforts, such as identification of appropriate test translators and judgmental designs for evaluating translation quality. Furthermore, three potential sources of translation bias (construct, method, and item bias) will be discussed. In doing so, statistical methods for evaluating measurement invariance for each source of bias will be presented. When presenting statistical methods, which will include factor analysis, generalizability theory, and differential item functioning, a brief description of each procedure will be provided taking into consideration practical issues, such as small sample sizes, where possible. In addition to quantitative methodology, the presentation will also include a brief discussion on how qualitative methods (e.g., think-aloud protocol and cognitive interviewing) can be applied to concerns of test adaptation quality.
Bio
Joseph A. Rios is an assistant professor in the Quantitative Methods in Education program at the University of Minnesota. He holds a PhD in educational measurement and psychometrics from the University of Massachusetts, Amherst. Prior to joining the faculty at the University of Minnesota, Dr. Rios worked four years in the research and development division at ETS. He is currently a consulting editor of the Journal of Educational Psychology, and his work has been published in peer-reviewed journals, such as Educational Measurement: Issues and Practice, Applied Measurement in Education, International Journal of Testing, and Educational Assessment.
One of his primary research interests is in understanding how to improve the assessment of linguistic minorities. This interest stems from the increasing population of English language learners in the U.S. public education system as well as the desire by countries to compare international student achievement (e.g., PISA). To assist in confronting this issue, Dr. Rios' research focuses on validity concerns around testing accommodations, such as linguistic adaptation of test content.