Quality Matters FAQs

Two week Introduction to QM and two and three-week Applying the QM Rubric (APPQMR) workshops are periodically offered. The workshops are totally Online (asynchronous). If you would like learn more information about the workshops (e.g. approximate number of hours spent), please visit the Quality Matters website.

To be eligible to be a Waubonsee Reviewer, you are required to complete the APPQMR Workshop. If it is your first time taking the workshop, the fee is covered.

Quality Matters recommends that a course is run at least twice before being reviewed using the Quality Matters Higher Education rubric.

Please note that internal reviews at Waubonsee using the QM HE rubric do not result in QM Certification. You can learn more about the HE Specific Review Standards by attending the Introduction to QM or APPQMR workshops.

Both during and after the workshops, you can use the QM Self-Review tool on MyQM Portal to evaluate your own course. The Self-Review is totally confidential. Please contact Faculty Development and Engagement with any questions you may have regarding QM standards or for assistance with course designing.

QM Peer Reviewers are not unique to Waubonsee and are able to review courses for other colleges and universities.

QM Peer Reviewers are considered private consultants and must have a clear understanding of the QM rubric. To be a QM Peer Reviewer, you must complete both the Applying the QM Rubric (APPQMR) workshop, the Peer Review Course (PRC), and submit an application. In addition, you need to be the instructor of record for an Online (asynchronous) course within the last 18 months.

The APPQMR workshop is the pre-requisit of the PRC. After the completion of the PRC, if you are eligible, you must complete the QM Role Application on MyQM portal to be listed as a Peer Reviewer on the QM website.

To be a Waubonsee Reviewer, you need to be a current Waubonsee instructor faculty and have completed the APPQMR workshop.

Emergency Remote Teaching and Sync Online is not considered an Online (asynchronous) teaching experience.

Quality Matters (QM) is an inter-institutional quality assurance organization. As a non-profit organization, they offer an internationally recognized faculty-driven Online Course Peer Review program founded on 4 underlying principles (or 4 C's):

1. Continuous (improvement)
2. (Student-) Centered
3. Collegial
4. Collaborative

The QM Higher Ed (HE) rubric emerged from research and instructional design principles for best practice. The rubric focuses on course design in courses that have an asynchronous online component.

Identical to Waubonsee's Online Course Peer Review in 2017-2020, QM applies only to Online (asynchronous) courses (.920-.929), hybrid courses (.950-.959), and Flex courses (.970-.979).

Note that like Face-to-Face courses, Emergency Remote Teaching and Sync Online is not a suitable candidate for QM certification.

An Instructional Designer through the Office of Faculty Development and Engagement acts as the Quality Matters Coordinator (QMC) and Course Review Manager (CRM) and will be available to assist through the review process.

QM focuses on the design of a course.

Design:

QM defines course design as the forethought and careful planning faculty put into a course.

Delivery:

This is the actual teaching of the course and the implementation of the design.

Example: What is design? It is the planning of a discussion board. Students are told when they should participate and how often they can expect the faculty to participate. What is delivery? It is how often faculty participate and what is being said to students.

QM suggests that reviewers take students' point of view to look for evidence that the rubric standard is met and not assume it is or isn't there.

The QM review program is faculty-driven and all reviewers are faculty in higher education.

For official QM certification, a review team must have three reviewers; the team must include at least one the following:

1. Master Reviewer who serves as the Team Chair 
2. Subject Matter Expert (SME)
3. External Reviewer

While QM suggests reviewers examine the course design from a student's perspective, there will be at least one person who has taught the subject and likely has taught the exact same course before.

The Higher Education rubric includes 44 Specific Review Standards (SRSs) divided in 8 categories:

1. Course Overview and Introduction
2. Learning Outcomes
3. Assessment and Measurement
4. Instructional Materials
5. Learning Activities and Learner Interaction
6. Course Technology
7. Learner Support
8. Accessibility and Usability

The rubric addresses three fundamental questions:

1. What do I want my students to learn?
   • Learning Outcomes
      
2. How do I know my students learn?
   • Assessment and Measurement
      
3. What do I do to help my students learn and succeed?
   • Course Overview and Introduction (Day 1 in the classroom)
   • Instructional Materials
   • Learning Activities and Learner Interaction
   • Course Technology
   • Learner Support (Relevant Institutional Resources)
   • Accessibility and Usability (Diversity, Equity, and Inclusiveness)

QM review standards have a quality goal of 85% or better.

For example: If there are 15 course outcomes and a reviewer believes 13/15 are measurable: For SRS 2.1, the reviewer will likely consider it "Met".

However, if another reviewer believes only 12/15 course outcomes are measurable, then the reviewer would consider SRS 2.1 "Not Met".

That is why QM requires a Review Team to have 3 Reviewers: Majority Rules. The point(s) awarded for each SRS is all-or-nothing.

Internal course reviews are scheduled to begin in Spring 2022. Faculty can opt-in to the QM  certification process when a course comes up for review.

Courses typically have at least three years between reviews. The previous review cycle (2017-2020) schedule will be followed as closely.

You are strongly encouraged to use the Self-Review tool to evaluate your course before making adjustments. The report will provide a good idea what could be improved before the review.

When a course is ready, the review starts with an application. In QM terms, a Course Representative (CR) is the person who initiates the application. A CR is usually the instructor who designed or helped design the course. Most of the process will be managed and completed by an Instructional Designer (Course Review Manager or CRM). Upon receiving an application the CRM will determine if Waubonsee should move forward with the QM Certification process:

1. Approve Course Review Application
2. Submit Course Worksheet (CR)
3. Approve Course Worksheet
4. Select Review Team
5. Confirm Reviewer Availability
6. Assign Reviewers to the Review Team
7. Create Tracking Sheet
8. Contact the Review Team Chair
9. Course Copy
   • Provide access/login to external/publisher materials (CR)
10. Provide Course Access to Reviewers
11. Pre-Review Conference Call (CR, QMC, all Peer Reviewers)

A review usually takes about 3 weeks. Well-designed and poorly-designed courses usually take less time.

Reviews begin after a pre-review conference call. Reviewers will evaluate SRS 2.1 (course outcomes) and SRS 2.2 (module outcomes) immediately. If two sets of outcomes don't match or either set does not meet the 85% threshold, the review will be paused until the outcomes are revised.

While a course does not need to be perfect for each SRS, it does need to meet the 85% threshold. Because SRS 2.1 and 2.2 are the first thing reviewers will examine, it is very important to have well-written outcomes!

Alignment is how course components work together to ensure students learn what you want them to learn.

QM defines alignment as "critical course elements working together to ensure that students achieve the desired learning outcomes". This is a reason why well-written outcomes are so important.

With a clear goal, it is easy to see if the course framework (content, assignments, and assessment) lead to what we say students should be able to do.

This will depend on whether a course meets QM requirements for certification. Each SRS uses the 85% threshold. The certification has 3 criteria. A course will earn the certification if all 3 criteria are fulfilled:

1. Alignment (SRS 2.1, 2.2, 3.1, 4.1, 5.1, and 6.1)
2. All essential 3-point SRSs are met (69 points)
3. A total score of 85 or higher (16+ points from other SRSs)

If a course does not initially meet all criteria, one of the following responses can be submitted:

• intent to make necessary improvements within a given time frame
• intent to make necessary improvements while petitioning for an extension
• ask the Team Chair to reconsider the review
• withdraw from further review

In any case, the Team Chair will send you a Final Report. All reviewers provide comments that explain where they found (or did not find) evidence for each SRS. If you intend to make adjustments, you can submit the Amendment Form up to two times within 14 weeks of the Final Report. The Team Chair will determine if the adjustments meet the requirements.

The certification does not distinguish if it is obtained before or after adjustments. When a course is certified, it will have access to the QM Certification Mark.

Once a course is officially QM-certified, QM provides Certification Mark, a digital image, for you and Waubonsee to embed in different web platforms, including the .edu site and Canvas. Each image is specific to a course and is not transferable.

The Certification Mark indicates that the course went through the QM Review process and successfully met the Review Standards. The image cannot be altered, however, a Widget Maker allows you to adjust the size and background color before displaying it.

If a course successful meets the Review Standards, the certification is good for up to 5 years. Recertification should take place before the end of the 5th year. Successful recertification and the subsequent use of the Certification Mark is good for another 3 years.

QM recommends that a course runs at least two times before the initial review. During the 5 years, adjustments for any component will likely be less than 20%, which is the threshold that triggers a recertification process.

Each course can only be recertified once. For recertification, unlike the initial review, only one reviewer is required. This reviewer must be a Master Reviewer, SME, and external to Waubonsee.

As an official QM Peer Reviewer (PR), your information is in the QM database. When colleges or universities have a need for a peer reviewer, they will search the database. If you are a match, they will contact you and gauge your interest. It is up to you to accept assignments.

All official QM Peer Reviewers are paid. The amount varies and is paid directly by the college or university that requested the review. Most reviews usually range from $100 to $250. The review team chair, who must be a Master Reviewer, typically gets $50 - $100 more.

Waubonsee Reviewers are paid a stipend by Waubonsee for each review.

After you have reviewed at least two courses as an official QM Peer Reviewer, you can apply to complete the Master Reviewer Certification (MRC). Being a MRC, you are eligible to be a QM Peer Review Team Chair.

Typically, the team chair get $50 - $100 more than the regular QM Peer Reviewer.