Evaluation using rubrics¶
Introduction¶
Most courses are evaluated by its learners.
Learners, however, are not teachers
and can be misguided on what teaching quality is.
For example, students values lecture higher than
active teaching, even though lectures results in lower
grades [Deslauriers et al., 2019].
An alternative is to use scientifically informed checklists of what good teaching is.
Research question¶
What are the checklists available?
POET-O by [Jia et al., 2025]¶
E.g. the POET-O observation and evaluation tool [Jia et al., 2025]:
| No. | Theme | Facet |
|---|---|---|
| 1 | Presentation and Activities | The lesson and learning activities are well organized |
| 2 | . | Planned student activities reflect appropriate lesson objectives |
| 3 | . | Depth and breadth of material presented appears appropriate to type of course, student level, and time dedicated to the topic |
| 4 | . | Learning activities provide opportunities for interaction that support active learning |
| 5 | . | Appropriate amount of content and course requirements are delivered |
| 6 | . | Discussion forum is interactive |
| 7 | . | Discussion forum and other learning tools promote critical thinking |
| 8 | Use of Technology | The instructor effectively uses audio/visual learning tools and technologies to support the learning objectives or competencies |
| 9 | . | Learning tools and technologies used promote learner engagement and active learning |
| 10 | . | Learning tools and technologies required in the module/unit are provided and/or readily obtainable |
| 11 | . | Access to links and external resources provided are functional |
| 12 | . | Learning tools and technologies used are current |
| 13 | ADA Compliance/Accessibility | Presentation and organization of materials facilitate ease of use |
| 14 | . | Information is provided regarding accessibility of all learning tools and technologies required in the lesson |
| 15 | . | Alternative means of access to lesson materials in formats that meet the needs of diverse learners are provided |
| 16 | . | All materials are designed to facilitate readability |
| 17 | Assessment | Planned assessment strategies measure the stated learning objectives or competencies |
| 18 | . | Specific and descriptive criteria are provided for the evaluation of learners’ work, i.e. rubrics etc |
| 19 | . | Students are given ample time to complete the assignments and assessments for this lesson |
| 20 | Instructor | Instructor provides an overview of what is planned for the lesson |
| 21 | . | Instructor appears well prepared for lesson |
| 22 | . | Instructor establishes the relevance of information |
| 23 | . | Lesson content is up to date and there is evidence that instructor is knowledgeable about topics presented |
| 24 | . | Content is explained clearly, providing examples when appropriate |
| 25 | . | Instructor makes connections with prior learning (from previous lessons and courses) when applicable |
| 26 | . | Instructor facilitates productive discussions related to content |
| 27 | . | Instructor encourages critical thinking and expression of divergent opinions or conflicting views when appropriate |
| 28 | . | Instructor contributes to discussion, providing encouragement, reflection, and corrective feedback |
| 29 | . | Instructor provides periodic summaries of the most important ideas and ties things together at the end of the lesson |
| 30 | . | Instructor uses active learning techniques |
| 31 | . | Instructor treats students respectfully, offering encouragement in public ways and does not criticize student publicly |
| 32 | . | Instructor utilizes diverse methods of instruction to accommodate all learners |
[Lee et al., 2020]¶
Or, from [Lee et al., 2020]:
| No. | Theme | Facet |
|---|---|---|
| 19 | Factor 1: Learning Activities & Materials | Assessments and activities are consistent with the course objectives and resources. |
| 22 | . | Activities provide students with opportunities to receive feedback early and frequently, specifically in preparation for high stakes assessments. |
| 23 | . | Course includes assessments and activities that are problem-centered or application-oriented in nature. |
| 24 | . | Students are encouraged to integrate new concepts into regular practice and understanding through demonstration, reflection, creation, or similar activities. |
| 25 | . | Resources & materials support learning objectives. |
| 26 | . | Resources & materials are sufficient for students to learn the subject. |
| 27 | . | Resources, materials, and instructor interactions activate students’ prior learning and experiences while introducing new concepts. |
| 43 | . | Tools and media support the learning objectives. |
| 44 | . | Tools and media are appropriately chosen and appropriately varied to enhance student interactivity with course content. |
| 48 | . | Course provides additional tutorials/resources as needed to accomplish objectives. |
| 1 | Factor 2:Course Introduction & Design | Upon first entering the course, students can easily find the course syllabus and introductory materials. |
| 2 | . | The progression of course content and activities is easy to find, clearly outlined, and appropriately segmented into units or modules. |
| 3 | . | Course appears visually clean, consistent, and appealing on the home page and throughout. |
| 4 | . | A course introduction orients student to the course environment and suggests the relevance of course materials and activities to students and/or program goals. |
| 49 | Factor 3: Learner Support | Course provides technical support services link/description. |
| 50 | . | Course provides academic support services link/description. |
| 51 | . | Course provides student support link/description. |
| 13 | Factor 4: Learner Engagement & Interaction | Provides clear expectations for student response, engagement, and participation. |
| 14 | . | Provides clear expectations for student etiquette in participation. |
| 34 | . | A means for making course announcements is clearly available and used regularly to encourage student completion and participation and to connect course content with current events and research. |
| 35 | . | Course design fosters interaction with other students. |
| 36 | . | Course design fosters interaction with content. |
| 37 | . | Appropriate synchronous or asynchronous means are provided for students to ask questions and receive answers from the instructor and/or students. |
| 16 | Factor 5: Learning Objectives | Objectives are clearly stated. |
| 17 | . | Objectives are measurable. |
| 18 | . | Objectives are consistent with the course material/assessments/assignments. |
| 5 | Factor 6: Course Facilitation | Course has an instructor introduction |
| 6 | . | Students have an opportunity to introduce themselves. |
| 12 | . | Course fees, if any, are explained. |
| 33 | . | Course design fosters interaction with instructors. |
| 7 | Factor 7: Course Information | The course grading policy is clearly stated. |
| 9 | . | Course technology requirements are addressed up front, if applicable. |
| 10 | . | Textbook information and other materials requirements are provided. |
| 38 | Factor 8: Course Technology | Course has a statement directing students with ADA-documented disability to the DRC for reasonable accommodations as needed. |
| 45 | . | Tools and media are as easy to use as is reasonably possible. |
| 46 | . | Tools and media are sufficiently compatible with web and other applicable standards. |
| 15 | Factor 9: Course Management | Syllabus addresses course-appropriate policies, including academic honesty, harassment, withdrawal and I-grades, and the student grievance process. |
| 20 | . | Appropriate pacing mechanisms (due dates, reminders, follow-ups) are used to ensure timely student completion and regular engagement. |
| 21 | . | Specific descriptive criteria are provided for the evaluation of student’s work and participation, ideally in the form of a rubric. |
References¶
[Deslauriers et al., 2019]L. Deslauriers, L.S. McCarty, K. Miller, K. Callaghan and G. Kestin, Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom, Proc. Natl. Acad. Sci. U.S.A. 116 (39) 19251-19257, https://doi.org/10.1073/pnas.1821936116 (2019).[Fernandez and Mateo, 1992]Fernandez, Juan, and Miguel A. Mateo. "Student evaluation of university teaching quality: Analysis of a questionnaire for a sample of university students in Spain." Educational and Psychological Measurement 52.3 (1992): 675-686.[Jia et al., 2025]Jia, Yuane, et al. "Validation of a peer observation and evaluation tool for online teaching in the US: Y. Jia et al." Educational technology research and development 73.1 (2025): 615-639.[Marsh, 1984]Marsh, Herbert W. "Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility." Journal of educational psychology 76.5 (1984): 707.