The purpose of this guide is to assist educators in evaluating the technical quality of a social and emotional learning (SEL) assessment by identifying what types of information and evidence a developer should provide for an assessment and determining if the assessment is appropriate for local plans for interpretation and use. Technical (psychometric) considerations addressed in this guide are based on the 2014 Standards for Educational and Psychological Testing published by the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME).
What is the purpose of the guide?
The purpose of this guide is to prepare assessment users to know what questions to ask regarding the availability of information and empirical evidence that may support the intended interpretation and use of an SEL assessment for their student population. The extent to which the technical evidence for an SEL assessment addresses these questions will assist informed selection decisions that will translate into more valid interpretations and uses.
Who are the intended users of this guide?
This guide is for educators tasked with the selection and use of an SEL assessment. Ideally, a group of educators and experts with relevant insights about the content to be assessed and the use of the resulting information to guide action will be involved in the evaluation. Although specialized psychometric expertise can be helpful, this guide was written for assessment users who may not be experts in the technical details of assessment development.
What types of interpretations and uses of SEL assessment does the guide address?
This guide is applicable to situations where SEL is measured to provide feedback and improve instruction and programs. The guide is not intended for situations where SEL assessments are being used for accountability or in consequential decision making at a group or individual level.
Consequential decisions at an individual student level would involve measuring student learning to screen or diagnose students in need of additional services or intervention or to identify students with a mental health concern. If an SEL assessment will be used for these types of consequential decisions, educators should consult with school district professionals who have received appropriate training and who hold the licenses or certifications necessary to conduct clinical evaluations of children for mental health or special education intervention, mental health diagnosis or special education classification. These professionals should also be well-versed in the tenets of the Standards for Educational and Psychological Testing (AERA, APA, NCME, 2014).
If an SEL assessment will be used to make high-stakes decisions about a school or program, it is highly recommended that evidence be carefully scrutinized with the assistance of someone with psychometric expertise and an understanding of the context in which the decisions will be made.
What do you need to know to use this guide?
The importance and relevance of different types of technical evidence are based on what SEL competencies you are wanting to measure; how you intend on using assessment results; your local setting and student population; and the format of the assessment, including how it is administered and scored.
For each consideration, the guide outlines what documentation the assessment developer should provide, what the test user should do with that documentation, some explanation and examples, and what to do if an SEL assessment does not meet the consideration. Information needed to answer these questions can be found on assessment developers’ websites, technical manuals, administration and scoring manuals or even in published technical evaluations of SEL assessments.
It is likely that most SEL assessments will not meet all considerations in this guide. The guide was written to address best practices rather than common practices so users should weigh the pros and cons of using an SEL assessment that may not meet one or more considerations. If an SEL assessment does not meet one or more considerations listed in this guide, it still might be appropriate to use that assessment if caution is used in interpretation and use unless the assessment will be used for high stakes decisions about individuals, schools or programs.
This guide is divided into four parts and all four parts should be applied when evaluating the technical quality of the SEL assessment.
- Does the assessment effectively measure the SEL competencies of interest?
- Does the SEL assessment provide credible evidence for your intended uses?
- Is the SEL assessment relevant for your students and your setting?
- Does the SEL assessment address issues related to administration, scoring, and the assessment format?
We encourage educators to field test these guides and share suggestions, questions, or comments about their usability and guidance with Dr. Jessica L. Jonson at jjonson@buros.org.