Assessment Development

The EM2 Project conducted research to develop and validate each diagnostic assessment

To develop diagnostic assessments that will support teachers’ efforts to identify student misconceptions, the EM2 Project used an iterative process that drew on the expertise of many individuals to develop each assessment.

An assessment is “valid” when it accurately measures what it is supposed to measure. The EM2 Project took several approaches to establish the validity of each assessment. These included using a principled and systematic approach to designing each assessment, establishing the content validity of each assessment, and examining the convergent validity of each assessment.

The project used components of Susan Embretson’s cognitive design system framework to develop each assessment (1998) . These include:

  • clearly articulating what we want to accomplish with each assessment,
  • identifying relevant features in the “task domain” (i.e., what are we asking students to do,
  • developing a cognitive model for the assessment (i.e., what are the different types of thinking in which we think think students will engage to answer the items),
  • generating items according to the cognitive model, and
  • evaluating the cognitive model.

  • This is a multi-step process through which we draw on the expertise of EM2 Project team members, as well as experts in the field who have deep knowledge about how students understand rational number concepts.

    Embretson, S. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning.Psychological Methods, 3, 380-396.