Is the SEL assessment relevant for your students and your setting?

← Back to Evaluating the Measurement Quality of SEL Assessments Download PDF

Assessment developer should…	Test user should…	Explanation	What to do if an assessment does not meet this criterion?
Identify the intended population for the assessment and clearly articulate if there are any inclusion or exclusion criteria.	Select an assessment that is intended for the key demographics (e.g. age/grade) of the local student population to be assessed.	Use assessments only with individuals who are demographically representative of the intended population. For example, Do not use an assessment developed for Grades 9 and up if the intended population of the assessment is middle or elementary school students Do not use an assessment with English Learners (ELs) if a developer indicates that the assessment is not appropriate for those students.	If the intended population for the assessment does not align with the key demographics of the local student population to be assessed, look for another assessment.
Provide a rationale and evidence that what and how SEL competencies are measured is developmentally appropriate for the grades/ages of students in the intended population.	Review the rationale and evidence to determine if the assessment is developmentally appropriate for the grade/ages of students in the local population.	Developmental appropriateness is particularly important if an assessment will be used to track SEL competency development over ages or grades. Student development of SEL competencies can differ not only because: Different SEL competencies become important at different developmental stages. Ways in which those SEL competencies are demonstrated or displayed changes over time. An assessment developer should address these developmental considerations when developing and validating the assessment.	If there is an insufficient rationale or evidence that an assessment is developmentally appropriate for the grades/ages of students in the local population, use for the grades/ages for which it would be appropriate or find another assessment.
Indicate the reading level and linguistic competency needed by respondents.	Determine if respondents will have appropriate levels of reading and linguistic competence.	The reading level and linguistic complexity of an assessment is not only important to consider in terms of students but for other respondents as well. For example, if a parent report would require a sixth grade reading level and English proficiency, ensure that most if not all parents will meet those requirements; if not, determine how to accommodate the participation of parents who do not meet those requirements.	If an assessment developer does not specify reading or linguistic competency needed by respondents, ask the assessment developer for more information, or have a reading specialist/ELL coordinator review the assessment to determine if it is appropriate for the local population.
Specify the availability of language and ability accommodations are available. For available accommodations, provide guidance on when to use the accommodation and how to administer and score it.	Determine if accommodations for students or respondents in the local population are available.	If the setting has a linguistically diverse student population or a sizable number of students with identified disabilities, the availability of accommodations would allow these students to participate. Seeking out the availability of multi-language versions or forms for students with disabilities would be another option.	If needed accommodations are not available, ask assessment developer for more information. If adequate accommodations do not exist, do not use the assessment for relevant students, or seek out experts who can assist in identifying accommodations that would remove barriers for these students but not change the SEL competencies measured.
Use a culturally representative panel to review SEL competencies measured by the assessment to determine if how those SEL competencies are measured are relevant for different cultures.	Review the demographics and findings of the panel to insure individuals from cultural groups represented in the local setting and student population are included and the assessment will fairly assess SEL competencies for those cultural groups.	How an SEL assessment defines and measures competencies may not be relevant to respondents from different cultural groups because the value of those SEL competencies or how they are represented may vary. Cultural differences are an important consideration when developing SEL programs and assessments along with systematic review and/or empirical studies to ensure they are not culturally biased. For example, Review panels should include members of each relevant cultural group or people either who work with or are familiar with those groups. Comments from such individuals should be considered seriously.	If the SEL competencies addressed by the assessment have not been reviewed and approved by a culturally representative panel, ask a panel that represents cultural groups in local student population about the relevancy of the SEL competencies, or do not use the assessment for making decisions about unrepresented cultural groups.
Have a diverse panel familiar with the needs of different students review the content and format of the assessment for bias, sensitivity, and accessibility. Document whether that panel found with a high level of agreement that assessment is unbiased, sensitive, and accessible. If the panel identifies items or format as biased, insensitive, or inaccessible, describe how those issues were addressed	Review the demographics of the panel and the findings of the panel to determine if the review is applicable to local setting and student population and if any bias issues were raised that might be a concern for the local student population.	Individuals of different backgrounds or individuals who are aware of capability differences among students should be involved in the development and review of SEL assessments. This includes: Review panels representing different racial/ethnic groups, ages, gender, individuals with disabilities, etc. Reviewing items for topic and wording that could be unfair or ratings of students by individuals who might have an unconscious bias.	If the developer has not used a panel that is representative of the local student population to review for bias, sensitivity, and accessibility, ask a local group that is familiar with the needs of different students in the local population to review the assessment and its items. If there is insufficient documentation that an assessment will be fair for specific demographic groups, do not use the assessment for those demographic groups.
Provide empirical evidence that responses to items/tasks and reported scores are not significantly different for students with similar levels of SEL competency from different demographic groups (e.g. race/ethnicity, language, and gender). If statistical differences exist, indicate actions taken to understand those potential differences better.	Review the evidence provided to determine if the assessment addresses key student groups in the local population and if it raises fairness concerns for individuals from those groups. Demographics to consider include gender, race/ethnicity, socioeconomic status, and language background.	Assessments can consist of items/tasks that do not function the same way for different group of students or differences between relevant subgroups on reported scores. At the item, measurement invariance studies (e.g. differential item functioning or multigroup confirmatory factor analyses) gather evidence that assessment items performs the same way for different groups of students. If studies find a lack of measurement equivalence, a follow-up study determining whether differences are potentially due to bias should occur before a user can credibly use the assessment for measuring the SEL competencies of those diverse groups. Those conducting the analyses must also be aware of the assumptions of the procedures and number of individuals needed to conduct those analyses to avoid misinterpretation of results. At the score level, differential prediction is a common method used to determine through regression analysis whether there are differences between relevant subgroups on reported scores.	If there is insufficient evidence that students with similar levels of SEL competency from demographic groups respond at the item/task or score level similarly, do not report and compare the scores from subgroups, or find another valid assessment for student populations that are demographically very diverse. If there is evidence of lack of equivalence at the item or score level that was not addressed by the assessment developer, do not interpret and use assessment results for those subgroups especially if they are a key group in the student population.

← Back to Evaluating the Measurement Quality of SEL Assessments Download PDF