This page shows the Evaluation Toolkit review criteria currently under consideration as of March 2005. These criteria have not yet been used.
Presents valid (correct) evaluation concepts and models for ESE evaluation
Addresses concepts, models, and skills useful for ESE evaluation
(Examples: inquiry-based teaching, higher-order thinking skills, Earth
Systems concepts, geoscience concepts)
Degree to which tool is usable as stand-alone
Author has provided annotations about use of tool
Degree to which other information is provided or resources indicated
Audience with which validated
Proven to measure what it intends to measure-need rubric for this.
Science Education Concepts
Scientific factual accuracy
Higher order scientific skills or inquiry-based
No conflict of interest or bias
Addresses diversity issues
EFFECTIVENESS AS EVALUATION TOOL
Anecdotes, questionnaires/surveys, interviews, observations, portfolios, tests, user logs, etc. Sample data collection rubrics, etc.
Indicates type of data collection required
Indicates knowledge and skills needed for this type of data collection
Provides information for finding people with appropriate skills
Provides list of resources for acquiring data collection knowledge
Indicates data handling issues
Indicates type of analysis suitable for this data -- e.g., t-tests, chi-square for comparisons; coding for interviews, etc.
Indicates tools needed for analysis -- statistics software, etc.
Indicates knowledge and skills needed for this type of analysis
Provides list of resources for acquiring skills http://www.soc.surrey.ac.uk/sru/sru.html
Provides list of resources for finding people who can conduct the analysis
Indicates goals for which this tool would be a suitable resource
Students' ability to conduct inquiry learning
Student motivation, engagement
Measuring achievement of content standards
Tool for measuring higher order thinking
Usability of online resources
Open-ended testing (what do students learn? rather than did they achieve the goals?)
Type of evaluation information
Deeper understanding about a particular teaching method's effects
Assessment of a program's success
Generalizability to larger audiences