Evaluation Toolkit Review Criteria

Evaluation Concepts

• Presents valid (correct) evaluation concepts and models for ESE evaluation

• Addresses concepts, models, and skills useful for ESE evaluation (Examples: inquiry-based teaching, higher-order thinking skills, Earth Systems concepts, geoscience concepts)

Supporting information

• Degree to which tool is usable as stand-alone

• Author has provided annotations about use of tool

• Degree to which other information is provided or resources indicated

Validation

• Audience with which validated

• Proven to measure what it intends to measure-need rubric for this.

Science Education Concepts

• Scientific factual accuracy

• Authentic assessment

• Higher order scientific skills or inquiry-based

• No conflict of interest or bias

• Addresses diversity issues

Data Collection

• Indicates type of data collection required

• Indicates knowledge and skills needed for this type of data collection

• Provides information for finding people with appropriate skills

• Provides list of resources for acquiring data collection knowledge

• Indicates data handling issues

Analytical method

• Indicates type of analysis suitable for this data -- e.g., t-tests, chi-square for comparisons; coding for interviews, etc.

• Indicates tools needed for analysis -- statistics software, etc.

• Indicates knowledge and skills needed for this type of analysis

• Provides list of resources for acquiring skills http://www.soc.surrey.ac.uk/sru/sru.html

• Provides list of resources for finding people who can conduct the analysis

Educational goals

• Indicates goals for which this tool would be a suitable resource

  • Students' ability to conduct inquiry learning

  • Student motivation, engagement

  • Measuring achievement of content standards

  • Tool for measuring higher order thinking

• Pilot testing

  • Usability of online resources

  • Open-ended testing (what do students learn? rather than did they achieve the goals?)

• Type of evaluation information

  • Deeper understanding about a particular teaching method's effects

  • Assessment of a program's success

  • Generalizability to larger audiences