Evaluating OER

Learning Objectives

At the end of this chapter, you should be able to:

  • Describe three aspects of an OER that should be assessed before use.
  • Explain why it is necessary to assess an OER’s adaptability.

Ensuring the quality of course materials is, obviously, of paramount importance.  One of the ‘myths’ around OER are that they are of lower/inferior standards than traditional, paid resources.  In fact, most OERs are high quality.[1]

Who Reviews OER?

You could argue that we have been crowdsourced quality control is a key feature of academia in the form of peer review.  We rely on our discipline-specific peers and other experts in the field to ‘vet’ content in the form of peer review.  In terms of OER, some are vetted through a formal peer review process; however, many times, the process is much more informal.  Many OERs are not vetted at all and the quality control aspect is up to the faculty member wishing to adopt or adapt the content.

Many publication platforms are taking steps to encourage review.  For example, the Open Textbook Library (OTL) encourages faculty members to assess open textbooks.

Blueprint for College Success Text

Some of the evaluation criteria listed below are universal, and others (such as Adaptability and Modularity) are specific to OER.[2]

We’ll be exploring these criteria for evaluation in our learning community in more depth and you’ll be applying them to a selected OER.

Clarity, Comprehensibility, and Readability

The most ubiquitous standard on this list is also the first: can the material you are considering be read and understood by your students? Although it might seem like a simple question, it is a necessary obstacle to confront when adopting a new resource for your course.

Considerations

  • Is the content, including any instructions and exercises, clear and comprehensible to students?
  • Is the content consistent with its language and formatting? (e.g. key terms are bold)
  • Is the content well-organized in terms of sequencing and flow?

Content and Technical Accuracy

The accuracy of the content you use is also a major component of its usability in the classroom. Be sure to check for technical errors such as broken links or typos. In most cases, content accuracy will not be an issue, but some older resources may require updates.

Considerations

  • Is the content accurate based on your expertise?
  • Are there any factual, grammatical, or typographical errors?
  • Is the interface navigable for students?

Adaptability and Modularity

Because of their open licenses, OER permit a wider range of (re)use than most traditional educational content; therefore, it is important to keep in mind how your chosen OER can be adapted. Modularity, or the ability to be broken up into smaller pieces easily, is one feature of an OER that should be preferred when possible. When creating OER, using clear chapter and unit breaks can help other instructors adopting or adapting your resource for their own courses.

Considerations

  • Is the resource in a file format which allows for adaptations, modifications, rearrangements, and updates?
  • Is the resource easily divided into modules, or sections, which can be used or rearranged out of their original order?
  • Is the content available under a license which allows for modifications?

Appropriateness and Fit

Although there may be OER available in your field, some resources may require minor edits or additions. Keep in mind that the open licenses of OER mean that they can be edited or even combined with other resources. This can be particularly useful if you would like to adopt a chapter from one OER for the first unit of your course but prefer alternate resources for other units.

Considerations

  • Is the content presented at a reading level appropriate for your students?
  • How does the content align with your course learning objectives?
  • Is the content level appropriate for use in your course?

Accessibility

No matter what resources you plan to adopt, accessibility should always be a part of your assessment process. Many published-provided homework products are not accessible to students and can cause unexpected issues. Similarly, some OER may not be optimized for students with visual or auditory impairments. We’ll be talking more about accessibility and DEIA in an upcoming session.

Considerations

  • Is the content accessible to students with disabilities through the compatibility of third-party reading applications?
  • If you are using Web resources, does each image have alt text that can be read? Do videos have accurate closed-captioning?
  • Are students able to access the materials in a quick, non-restrictive manner?

If you would like a personal copy of these considerations, visit or download them through the Evaluating OER Checklist in Google Docs.

Additional Rubrics*

* Note: these resources are organized from very basic to complete evaluation rubrics.

Faculty Guide for Evaluating Open Education Resources, BCOER Librarians
This is a very brief checklist with some suggestions to think about before using an OER.

List of Criteria for Evaluating OER -Affordable Learning Georgia 
You can also download the list as a modifiable .doc.

[Summarized] Rubrics for Evaluating Open Education Resources Objects

This 2-page rubric is a synthesis version of the eight (8) separate rubrics for the evaluation of OERs created by ACHIEVE.org.

It is meant as a ready reference for quick evaluation of an OER.

Equity Through OER Rubric
Rubric with “[a]n overarching goal is to enable users to integrate OER in equitable ways across higher education leading to quality and equitable student access, outcomes and success.”

Rubrics aligned to and adapted from the Achieve rubrics

Comprehensive OER Evaluation Tool 
A detailed, iterative rubric for OER evaluation.

 


  1. https://link.springer.com/content/pdf/10.1007/978-981-19-0351-9_43-1.pdf
  2. Attribution: These criteria are from the Affordable Learning Georgia "Selecting Textbooks" webpage have been used with permission from the creator.

License

Icon for the Creative Commons Attribution 4.0 International License

University of Virginia OER Learning Community Guide Copyright © 2023 by Bethany Mickel and Emily Scida is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.