Does the assessment effectively measure the SEL competencies of interest?

Assessment developer should…Test user should…ExplanationWhat to do if an assessment does not meet this criterion?
Clearly identify and define which SEL competencies the assessment measures.Determine if SEL competencies of interest aligns with the SEL competencies measured by the assessment.

In order to determine whether an assessment will measure the SEL competencies of interest, those competencies must be stated in measureable terms that not only identify the competency of interest but also what a student will know, do, and/or understand as a result of achieving the SEL competency. 

SEL assessment should measure not only the competency of interest but also how students are expected to express that competency. 

  • SEL competencies of interest could be more general (e.g. intrapersonal or interpersonal skills) or more specific (e.g. growth mindset, self-efficacy, collaborative problem solving).
  • Expression of SEL competencies might also differ such as demonstrating awareness (e.g. mindsets, knowledge, beliefs) or applying skills (e.g. learned abilities). 

For example, if students should demonstrate problem-solving skills, the assessment should measure how students use and apply those skills not whether they are aware of the importance of those skills.

If a measure addresses none of the specific or general SEL competencies of interest or very few, find another assessment.

If the measure addresses some but not all SEL competencies of interest, look for a more comprehensive measure or a second measure to supplement information gathered.

If an SEL assessment does not provide a clear description of SEL competencies measured, do a formal review of items to make that determination or find another assessment that does measure the SEL competencies of interest.
Identify why respondents for the assessment are in the best position to assess students' SEL competencies.Consider whether the respondent for the assessment is the best source for assessing the SEL competencies of students in the local population.

If SEL competencies of interest involve attitudes, beliefs, or growth mindsets, respondents could be students reporting on their own SEL competencies.

If the SEL competencies are behaviors, respondents should be individuals who know the students well enough to assess their SEL competencies.

If the SEL competencies are knowledge or mental processes, responses should involve students demonstrating those SEL competencies through a direct assessment or performance task.

If the intended respondents for the assessment are unfamiliar or unable to accurately assess SEL competencies in the local student population, find another assessment.
Use a representative panel of content experts to develop and/or review items/tasks and scoring protocols to ensure that the assessment addresses SEL competencies sufficiently and appropriately.Conduct a local review of assessment items/tasks and scoring protocols to determine if those items sufficiently and appropriately address the competencies and outcomes for the local SEL program.

Clear and detailed specifications of the SEL competencies measured is important when developing not only tasks but also scoring protocols to ensure alignment between those defined specifications and the items/tasks and scoring protocols.

Assessment developers can use expert review, an assessment blueprint, and/or mapping of items/tasks onto scores, to demonstrate that items/tasks represent a cross-section of competencies measured. For example,

  • Asking individuals with emotional regulation expertise to review items from an emotional regulation scale and indicate the extent to which each item aligns with the SEL competency and if the set of items overlook important aspects of the SEL competency.
  • Having a group of content experts review the number and content of items/tasks to determine if the assessment coverall all measured SEL competencies sufficiently.

As a general guideline,

  • Selected-response assessments should have at least three to five items for each competency measured. 
  • Performance assessments typically involve a smaller number of tasks but that could hinder the generalizability of the scores if there is too broad of a set of SEL competencies measured.  
If the developer does not document that the assessment sufficiently and appropriately addresses SEL competencies, conduct a local review with appropriate content expertise or find another assessment that provides this type of documentation and evidence.
Provide empirical evidence that items/tasks used to measure each competency are more highly related to each other than to items that measure other competencies (internal structure).Determine if evidence supports that items/tasks used to measure SEL competencies are more highly related to each other than they are to items that measure other competencies.

If an assessment claims to measure three competencies, there should be higher correlation among items/tasks that measure the same competency than among items/tasks that measure the other two competencies.

Statistical analyses are used to support the assumption that unique rather than redundant information about each competency exists and these analyses typically require large sample sizes.  For example,

  • For selected-responses assessments, confirmatory factor analysis can provide evidence that items load significantly on to factors that represent the different SEL competencies measured by the assessment.  
  • For performance assessments, generalizability may be used to demonstrate that variability exists across different tasks.
If evidence of internal structure does not support that items/tasks used to measure different competencies are unique rather than redundant, use caution when reporting, interpreting, and/or using scores for individual competencies.