Metadata Collections & QA

Resource Quality Guidelines

Purpose

The Digital Library for Earth System Education (DLESE) established formal resource quality guidelines to:

  • Explain accessioning criteria in order to assist collection builders in identifying and selecting quality resources
  • Help users understand the level of quality of information they will find in DLESE collections and appreciate the the service is selective and quality controlled
  • Inform creators of practices for developing quality resources

In order to be accessioned in DLESE, resources (the actual content, not metadata) are:

1. Relevant to Earth system education

The DLESE Collection Scope and Policy Statement defines Earth system education. Resources in the library should fall within this scope. DLESE favors resources designed for teaching, research and learning about the Earth system. Background material, references and other supporting resource types are not excluded if they are within DLESE scope.

Top

2. Scientifically accurate

DLESE resources are aligned with sound scientific principles and practices. This means

  • The review for scientific accuracy is made by qualified specialist(s) in the appropriate field of science and the qualifications of the specialists are documented and revealed
  • The science review is completed by people who are not directly involved with the design and production of the resource, that is, they are documented (shown) to be external to resource development
  • Errors of fact identified in the science review process must be revised before accessioning
  • To help learners and teachers understand the reliability of the various parts of the resource, the resource should distinguish between what is observation/fact and what is interpretation/hypothesis and should identify assumptions
  • Reference lists or bibliographies are present
  • Links to related web sites are present
  • Technical terms are defined (or a link to a glossary provided)

DLESE also recognizes there may be important exceptions to this practice. For example, historical documents, which represent scientific knowledge of a previous generation rather than "current scientific knowledge," can be important for learning the history and epistemology of science. Other examples include resources that express one point of view on a controversial topic and might be considered by some to be outside "sound scientific practices." Such resources can be valuable for teaching critical thinking skills such as evaluating viewpoints and weighing the effects of motives and biases.

Top

3. Attributed as to origin

Attribution may be to a person or persons or to an organization or both. The rationale for this recommendation is that knowledge of where a resource comes from--its author, sponsor or publisher--helps library users judge how much credence to give the resource. This means the following is known about the resource

  • Name of the resource creator (individual and/or sponsoring institution)
  • Contact information for the resource creator or maintainer
  • Resource creation or most recent update date (if possible)

4. Robust and function well as a digital resource (combined a DRC and resource quality practice)

  • Be free of conspicuous bugs, defects and non-working elements (links that don't work, graphics that don't display, applets that don't run) that inhibit intended use. (combined 2 bullets into 1)
  • Should work on current versions of major web browsers, under operating systems used widely in educational settings
  • Text and graphic elements should be printable

DLESE understands that not all web sites and software can be completely bug-free. Resources that are still under construction and others that do not meet this criterion may be more appropriate for work or developmental areas of the library.

Top

5. Pedagogically effective and documented

If appropriate, resources are reviewed for pedagogical effectiveness and documentation.

Documentation

  • Materials lists; links or contact information to acquire uncommon materials
  • Safety precautions
  • An estimate of time required if applicable and especially for labs, activities, etc.
  • Skills and understandings needed prior to working with the resource
  • Age or educational level for which the resource is recommended
  • Learning objectives
  • Commonly observed mistakes or misconceptions
  • Notes on instructional strategies
  • Alignment with educational standards

Effectiveness

  • Feedback from educators who have observed learners interacting with the resource
  • Feedback from learners who have learned by using the resource
  • Feedback from an expert in pedagogy
  • The instructional time required for use of the resource is commensurate with the importance and magnitude of the learning achieved
  • The level of difficulty is appropriate for the stated target audience
  • The instructional strategies build toward mastering the stated learning objectives
  • Student learning achieved by the use of the resource is assessable and the resource should assess whether the learning objectives have been met
    • Include assessment options with or within the resource
    • Include guidance to teachers such as answer keys or scoring rubric

Top

6. Important or significant

The resource deals with content, skills or understandings that are likely to be important to a significant fraction of teachers and learners in the DLESE community. While the resource itself may not state why the learning made possible by the resource is important or significant, one should be able to make a case for it. Possible reasons include that the resource supports:

  • Learning that is called for in national or state educational standards
  • Learning on a topic of societal importance
  • Mastering of foundational skills needed for advancement in a discipline or career
  • Understanding of a concept which is central to a discipline or Earth or environmental science
  • Understanding of linkages and interactions between or within portions of the Earth system
  • Other compelling reasons as articulated in the resource

Top

7. Complete in documentation

In addition to the already mentioned attribution to origin, pedagogical effectiveness and documentation and scientific accuracy, resources need to provide documentation in other areas relating to technical and data and model issues. That is:

Technical

  • Hardware needed to use the resource
  • Software, including plug-ins, needed to use the resource, including the links or contact information to acquire such software
  • Any known hardware or software incompatibilities
  • For software or complex resources, a clear and comprehensive user's guide
  • File sizes for downloadable materials or large viewable files

Data

  • Basic information about the instrument which collected the data and how it works (or links to such information)
  • For individual data sets, information about when and where the data was collected
  • For derived data products or merged data sets, links to the data archive or other source of description about how the data were processed or merged and information about when and where the data was collected
  • Links to sources of original data and tools to access the data

Models

  • Name of the model
  • Description or link to the description of assumptions underlying the model and how the model works

Top

8. Easy to use for teachers and learners

  • Navigation pathways in the resource should be self-evident and intuitive
  • Preparation time and effort required by the instructor should not be excessive, relative to the educational potential of the resource
  • Provides methods that ease the process of adapting to students' and teachers' individual differences, interests, abilities and needs.
    • Provide teacher-editable versions of student hand-outs (MS Word or PDF documents)
    • Provide several levels of difficultly or complexity in student activities
    • Provide links to maps or local data sets for use in different locations
  • Provide information about the ease of use, if possible
    • Feedback from educators who have observed learners interacting with the resource
    • Feedback from learners who have learned by using the resource
    • Feedback from an expert in usability of web resources

Top

9. Inspirational/motivational for learners

  • Provides information on inspirational or motivational aspects of the resource
    • Feedback from educators who have observed learners interacting with the resource
    • Feedback from learners who have learned by using the resource
    • Feedback from educational specialists familiar with the target audience
  • The resource strives to inspire or motivate learners to
    • Show curiosity about the topic or the resource
    • Ask questions or seek out further information
    • Perform at a high level (i.e. do their personal best work)

10. Free of advertising that is distracting or off topic for the learner

Resources should be free of advertising as much as possible, especially advertising that is irrelevant or interrupts or interferes with the use of the resource.

11. Free or low cost to educational users

DLESE favors resources that are free or available at low cost. Generally, subscription-based resources are avoided.

Top

Practices to avoid

  • Intrusive advertising
  • Information specific to one school or course, such as room numbers, office hours, etc.

Management

These guidelines provide interpretation of criteria within the DLESE Accessioning and Deaccessioning Policy. These guidelines are the responsibility of the Collections Committee.

Credits

This policy resulted from prior work by the:

  • DLESE Collections Committee
  • DLESE collections meeting in March 2003
  • DLESE Quality Workshop in June 2003
  • DLESE Quality Workshop in October 2004

Top

Last updated: 1-13-05