Metadata Collections & QA
Skip navigation Digital Library for Earth System Education
Digital Library for Earth System Education
Search tips

Evaluation Toolkit Review Criteria

This page shows the Evaluation Toolkit review criteria currently under consideration as of March 2005. These criteria have not yet been used.

CONTENT
1 (poor)
2
3
4
5 (best)

Evaluation Concepts

  • Presents valid (correct) evaluation concepts and models for ESE evaluation

  • Addresses concepts, models, and skills useful for ESE evaluation (Examples: inquiry-based teaching, higher-order thinking skills, Earth Systems concepts, geoscience concepts)

                     

Supporting information

  • Degree to which tool is usable as stand-alone

  • Author has provided annotations about use of tool

  • Degree to which other information is provided or resources indicated

          

Validation

  • Audience with which validated

  • Proven to measure what it intends to measure-need rubric for this.

         

Science Education Concepts

  • Scientific factual accuracy

  • Authentic assessment

  • Higher order scientific skills or inquiry-based

  • No conflict of interest or bias

  • Addresses diversity issues

         
 

EFFECTIVENESS AS EVALUATION TOOL

Anecdotes, questionnaires/surveys, interviews, observations, portfolios, tests, user logs, etc. Sample data collection rubrics, etc.

1 (poor)
2
3
4
5 (best)

Data Collection

  • Indicates type of data collection required

  • Indicates knowledge and skills needed for this type of data collection

  • Provides information for finding people with appropriate skills

  • Provides list of resources for acquiring data collection knowledge

  • Indicates data handling issues

              

Analytical method

  • Indicates type of analysis suitable for this data -- e.g., t-tests, chi-square for comparisons; coding for interviews, etc.

  • Indicates tools needed for analysis -- statistics software, etc.

  • Indicates knowledge and skills needed for this type of analysis

  • Provides list of resources for acquiring skills http://www.soc.surrey.ac.uk/sru/sru.html

  • Provides list of resources for finding people who can conduct the analysis

         

Educational goals

  • Indicates goals for which this tool would be a suitable resource

    • Students' ability to conduct inquiry learning

    • Student motivation, engagement

    • Measuring achievement of content standards

    • Tool for measuring higher order thinking

  • Pilot testing

    • Usability of online resources

    • Open-ended testing (what do students learn? rather than did they achieve the goals?)

  • Type of evaluation information

    • Deeper understanding about a particular teaching method's effects

    • Assessment of a program's success

    • Generalizability to larger audiences

         

EASE OF USE
1 (poor)
2
3
4
5 (best)
Clearly stated purpose               
Use of jargon or discipline-specific language          
Audience for which it was intended          
Audiences for which it would be useful          
Accessible for the intended audience (e.g. if for novice use, has a minimum of evaluation jargon)          
Organization          

TECHNOLOGY
1 (poor)
2
3
4
5 (best)
Accessible quickly with slow connections and processors               
Free of technical errors (e.g. bugs, broken links, dead ends, pictures that don't load, animations that don't work, etc..)          
Easily navigable and used          

CRITERIA TO CONSIDER
1 (poor)
2
3
4
5 (best)
Intellectual Property               
English/Spanish also good?          
Digital or easily digitized, or so good we could catalog a digital pointer          
Free or low cost a value          
Accessibility to various handicaps, such as color blindness, blindness          
Criteria Last modified 2005-03-28

Top

Last updated: 2007-03-08
Maintained by: Katy Ginger (support@dlese.org), DLESE Metadata