In higher education, online programs are chasing QM Certification. What does that mean?\u00a0Quality Matters is a lot of things: a non-profit, a methodology, a rubric.<\/p>\n
Instead of forcing instructors to all become instructional designers with QM training, all schools should have dedicated online education experts. Basic shells should be prepared by these instructional designers, supporting educators. Educators have enough to do without more responsibilities.<\/p>\n
If the IT\/ID teams want to adhere to QM methods, fine. But don\u2019t ask instructors to become designers, in addition to being subject matter experts.<\/p>\n
I dislike Quality Matters. It embodies many trends in higher education I dislike and resist, including: rubrics, checklists, processes over content, certifications, jargon,\u00a0<\/strong>acronyms, and commodification.<\/strong><\/p>\n Rubrics<\/strong> come in many forms, and most of them are flawed for a reason economists long ago recognized: you get what you measure<\/em>. If you tell students in a science course that \u00a020 percent of an essay grade is grammar and 20 percent is formatting, you\u2019ve suggested to them that 40 percent of a grade has little or nothing to do with subject matter. At the same time, rebalancing the rubric might cause students to ignore language and formatting.<\/p>\n When I\u2019ve been asked to use rubrics, I always include some \u201call or nothing\u201d statements. If you fail to meet the assignment objective, that trumps the points of the rubric. There are good and really bad rubrics, but I have yet to see a great rubric.<\/p>\n I love checklists<\/strong> and following processes.<\/strong> You cannot bake cookies without a checklist and a clear process. That\u2019s not the sort of situation in which \u201cprocess over content\u201d becomes a concern. Instead, I\u2019m talking about when we emphasize a formal process inflexibly and end up with content that\u2019s undesirable.<\/p>\n In computer programming courses I was taught to always flowchart a routine or function. The flowcharting received so much attention that students created beautiful charts\u2026 and broken code. That seems impossible, yet it happened time and time again. Flowcharts, design documents, procedural manuals, and other planning tools have value only if they support and contribute to a goal without getting in the way.<\/p>\n Change orders are a good example in programming. You could spend hours, days, or even weeks documenting a use change request for an enterprise project. By the time the change is approved, the user has realized a different function or feature better meets current needs. Companies that adopted strict procedures were slow to adapt. That\u2019s how \u201cagile\u201d coding methodologies became popular: they emphasize functionality over process.<\/p>\n Certification<\/strong> insanity makes credentials the only accepted measure of expertise. That\u2019s wrong, and most of us know it. Plenty of great instructional designers started before there were certifications in the field, much less degree programs in course design. Certified Peer Reviewer. Master Reviewer. Course Review Manager. QM offers plenty of certifications\u2026 all for a price. Someone benefits from the gatekeeping, especially those who have already endured the ritualistic hazing.<\/p>\n To make something a magical cult, it needs mysterious language. That\u2019s where the acronyms<\/strong> and jargon enter.<\/strong> If Quality Matters is \u201cQM\u201d and the staff is talking about GSs, SRSs, CBEs, APPs, MRCs, CPRs, and so on.<\/p>\n A quick intro to QM from the Quality Matters<\/a> website:<\/p>\n QM originated in 2003 as a Fund for the Improvement of Postsecondary Education (FIPSE) grant awarded to MarylandOnline, Inc. (MOL), titled “Quality Matters: Inter-Institutional Quality Assurance in Online Learning.” The purpose of the grant was to improve the quality of shared online courses by establishing an inter-institutional peer-reviewed quality assurance certification process.<\/p>\n Evolution<\/strong> Who could be against \u201cquality\u201d in education? And we\u2019re going to assure it! How? By developing a standardized, uniform approach to \u201cshared online courses\u201d that offered credit across multiple college and universities.\u00a0If everything is the same, across campuses, we\u2019re somehow closer to quality. Oh, but not the content. Trust us. We won\u2019t dictate the content, only the user experience with that content.<\/p>\n \u201cSelf-supporting\u201d means we need to earn money. Hello, membership fees. Hello, certification fees. But don\u2019t worry, no one is making any money.\u00a0I view QM as \u201cnon-profit\u201d in the same league as ETS (Educational Testing Services) and object to its approach to online education.<\/p>\n At the core of Quality Matters is the QM Rubric, which is used to evaluate online courses. There are 42 (ask Arthur Dent why) Specific Review Standards within eight General Standards. A 42-item checklist would be laughable, but I\u2019ve had to complete the forms and perform a mock evaluation. It\u2019s far too many items to be useful.<\/p>\n The eight General Standards in the Quality Matters Rubric are:<\/p>\n The \u201cCourse Worksheet\u201d is a lengthy document, used by evaluators (and ideally the instructor) to determine the strengths and weaknesses of an online course design. It took me several hours to complete a CW for a mock peer evaluation. That experience was enough to never take the process seriously. Technical communication scholars know that long questionnaires encourage people to take shortcuts and omit data.<\/p>\n The major headings for the Course Worksheet (CW) include:<\/p>\n I hate everything about QM, but I hate its embedded ableism most of all.<\/p>\n It required 61 presses of the tab key to reach the \u201cModule 5\u201d quiz that we were expected to evaluate in the QM course. Imagine striking tab 61 times. Again and again and again. Plus, trying to remember where you are in the other materials. When you leave the page or tab between windows to get between the homework form and the sample course, the darn navigation resets. Therefore, 61 tabs (or more, since you need to get into the quiz) again. And\u2026 again\u2026<\/p>\n And I missed the \u201cright\u201d link more than once with my palsy.<\/p>\n I decided to record my efforts and send them to the university IT department because I was so angry.<\/p>\n With adaptive technology, one of the quizzes required 2 hours and 20 minutes for me to complete. I passed (7\/8) but the physical and cognitive costs were inexcusable. For several hours, I experienced palsy attacks and seizures, leading to a severe migraine. The system is not accommodative of physical and cognitive differences.\u00a0The question I missed was the result of just giving up trying to navigate multiple tabs and screens to gather information.<\/p>\n QM is not, ironically, meeting universal design principles for actual users. They just checked off that they have captions or headings. Usability includes not forcing users to constantly switch about a screen.<\/p>\n Here is the expected schedule for the online training, from the QM schedule:<\/p>\n Almost 17 hours. I need at least twice the allocated time. This is not equitable, a core principle of online education. This clearly penalizes those not attending the other format and penalizes those with special needs even more.\u00a0Obviously, my anger is increasing the more I reflect on this process and the model that was presented.<\/p>\n I\u2019m not a fan of \u201cpedagogy models\u201d or diagrams suggesting hierarchies of learning stages. Yes, I have had to reference Bloom\u2019s Taxonomy in my lesson plans. I\u2019ve had to associate my course materials with various circles, triangles, and webs supposedly illustrating how we learn.<\/p>\n We do not learn following a linear path, or in stages represented in a triangle.<\/strong> That\u2019s nonsense. I realize proponents argue that models are merely starting points, but many new teachers are told, \u201cStudents cannot evaluate and analyze until they\u2019ve progressed upwards through the stages.\u201d I had to memorize the\u00a0mnemonic ESAACK: evaluation, synthesis, analysis, application, comprehension, and knowledge. The goal, the ideal, is evaluation. My education courses suggested most students never reach that tip of Bloom\u2019s Taxonomy.<\/p>\n At least Bloom\u2019s Taxonomy has something to do with how people learn.\u00a0Quality Matters has\u00a0absolutely nothing<\/strong> to do with how we learn. Nothing. It has everything to do with its 42-item rubric. As pedagogy scholar Jesse Stommel notes, administrators and regulators love models, even though there\u2019s no evidence that models do much. A great teacher might learn from models and then develop a personal approach.<\/p>\n Stommel offers an insightful critique of Quality Matters. He begins with an overview of models in general:<\/p>\n Not Taking Bad Advice: a Pedagogical Model<\/a> The text for my flipped keynote at Digital Pedagogy Lab 2020.<\/a><\/p>\n I’ve never created a model for online, digital, or\u00a0hybrid pedagogies. As long as I’ve been teaching, and as long as\u00a0I’ve been teaching teachers, I’ve encountered, been Quality Matters hides behind its complexity and mysterious language. As Stommel explains of such models:<\/p>\n Some of the more insidious models are fashioned as needlessly\u00a0complex in order to create a mystique of intellectual rigor. Even when\u00a0these models aren’t based in research, they are made to seem as\u00a0though they are.<\/p><\/blockquote>\n Pedagogy is the science<\/strong><\/em> and art<\/strong><\/em> of teaching. It\u2019s when a teacher adjusts lessons mid-class session to meet students where they are\u2026 to guide students towards where they need to be. Pedagogy requires knowing your students as people and yourself as a teacher. I have taught four sections of a class in a semester, and all four were different. That\u2019s pedagogy: applying research and best practices in a classroom.<\/p>\n Pedagogy is praxis, the intersection between the philosophy and practice of teaching. Best practices, which aim to standardize teaching and flatten the differences between students, are anathema to pedagogy. The most egregious example is the Quality Matters rubric, which the organization (a standalone non-profit as of 2014) calls the “QM Quality Assurance System,” which is said to “create a culture of continuous improvement” to “deliver the promise” of online learning. I have previously analyzed the marketing copy of Turnitin<\/a> and various learning management systems<\/a>. This work is not a mere exercise. When we’re deciding what tools we use, we should be looking careful at the discrepancies between what the companies say the tools do and what they actually do. At the points where these diverge, we begin to see the cracks in a tool’s promise<\/em>.<\/p>\n From the front page of the\u00a0Quality Matters Web site: “With online learning, everyone has a\u00a0goal. Learners need to improve and grow. You work to nurture them with Hold on here, you should be thinking, pedagogy cannot be a single design approach. One process cannot be \u201cbest\u201d for all groups of students or all teachers. Quality Matters promises the implausible.<\/p>\n When I\u2019ve watched colleagues with the most complex rubrics, longest checklists, and most detailed syllabi, I\u2019ve noticed they panic when anything deviates from the plan. Those instructors get lost in the process, at the expense of teaching content. We all know what happens with long online documents: \u201cTL\/DR\u201d (Too Long \/ Didn\u2019t Read).<\/p>\n If you have to tell readers something is clear\u2026 it is not clear.<\/p>\n Quality Matters is a special kind\u00a0of model, almost the polar opposite of Bloom’s, because QM is\u00a0decidedly not at all simple. It is, in fact, needlessly complex to the Teaching shouldn\u2019t be like engineering widgets. Quality Matters tries to be the ISO 9000 of online learning as if our schools manufacture outcomes. Documenting everything often improves nothing.\u00a0Quality Matters suggests, however, that documenting magically addresses problems with online course designs. Yet, the problems in education aren\u2019t addressed by QM. Instead, QM documents and reinforces what administrators want.<\/p>\n Stommel looks carefully at the language of the QM Rubric.<\/p>\n In\u00a0“The Trouble with Rubrics,”<\/a> Alfie Kohn writes, “Consistent and uniform standards are admirable, and maybe even workable, when we’re talking about, say, the manufacture of DVD players.”<\/p>\n QM\u00a0promises efficiency and objectivity, but actually creates more work and\u00a0merely provides cover for all the biases and subjectivity that continue\u00a0to exist (and are left unchecked) in spite of the rubric. The words “access,” “accessible,\u201d and “accessibility” do appear five times on the Quality\u00a0Matters rubric, but each use refers to either providing \u201cpolicies” and “statements,” or having accessible “text and image files” and “multimedia content” that\u00a0meet “the needs of diverse learners” (the only context Quality Matters feels like Henry Ford trying to tackle online education, and flailing about miserably. There\u2019s no innovation. There\u2019s no revolutionary insight within the rubric or other QM materials.\u00a0Let\u2019s all be the same. Teachers, let\u2019s all follow the same model. Students, learn this model and follow it. Learning will surely happen.<\/p>\n Our efforts toward\u00a0building community should be directed toward the students who need that\u00a0community the most, the ones most likely to be feeling isolated even Flexibility and trust are key principles of any\u00a0pedagogy worth its salt, but they are particularly important when\u00a0we’re in crisis. Right now, we need to focus on\u00a0“teaching the students we have, not\u00a0the students we wish we had.”<\/a> We don’t need more models for hypothetical students we haven’t yet met. And so, my pedagogical model has exactly one point:<\/p>\n Stop\u00a0looking for models and begin by talking to\u00a0students.<\/strong><\/p>\n About:\u00a0<\/strong>Jesse Stommel is co-founder of Digital Pedagogy Lab and Hybrid Pedagogy: The Journal of Critical Digital Pedagogy<\/em>. He has a PhD from University of Colorado Boulder. He is co-author of An Urgency of Teachers: the Work of Critical Digital Pedagogy<\/em>.<\/p><\/blockquote>\n Quality Matters repelled me. It is one of several reasons I\u2019m not returning to an adjunct post. Are there good practices within QM? Yes. But, there are also some horrible ideas I reject, such as constantly referencing learning objectives and outcomes throughout a shell. No student will page back and forth to decode \u201cLO 4.2\u201d in an assignment description. The course shell becomes a text-dense maze students won\u2019t read through. It is exclusionary, in practice.<\/p>\n As I wrote above, the training by Quality Matters was inaccessible. It was ableist, classist, and exclusionary.<\/p>\n The real accomplishment of QM is that it erects another barrier between students and teachers.<\/p>\n <\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" In higher education, online programs are chasing QM Certification. What does that mean?\u00a0Quality Matters is a lot of things: a non-profit, a methodology, a rubric.…<\/p>\n
\nInterest in the grant and the process it was creating spread quickly beyond MOL. When the grant ended in August 2006, it was determined that the work begun during the grant should continue. In September 2006, the Quality Matters grant became the Quality Matters Program, a non-profit, self-supporting program of MOL. In 2014, the Quality Matters Program began operating as Quality Matters, a non-profit organization with its own\u00a0Board of Directors.<\/p><\/blockquote>\n\n
\n
Accessibility as an Afterthought<\/h3>\n
\n
Pedagogy Models Stink, But QM Is NOT a Pedagogy<\/h3>\n
\nJesse Stommel
\nExecutive Director, Hybrid Pedagogy, a nonprofit supporting better teaching (2011 – present)
\n26 Jul 2020<\/p>\n
\nflummoxed by, or have cast off models. I have yet to see a single\u00a0pedagogical model worth its salt. And yet I watch how quickly models\u00a0spread. The neater and tidier the model, the more likely it seems to be\u00a0broadly adopted by an institution: Learning Styles, Bloom’s\u00a0Taxonomy, ADDIE, Scaffolding<\/a>, Design Thinking, Quality Matters, Andragogy, HyFlex. Lists, frameworks, Venn diagrams, rubrics, templates. Six principles of Andragogy, five stages of the ADDIE development process, six levels of Bloom’s Taxonomy, forty-two review standards of the Quality Matters Rubric<\/strong>. And, even when these models are thoroughly debunked, they continue to retain traction. According to the Association for Psychological\u00a0Science<\/a>, “No less than 71 different models of learning styles have been proposed over the years … But psychological research has not found that people learn differently, at least not in the ways learning-styles proponents claim.” And, yet, the\u00a0American Psychological Association found<\/a>, “in two online experiments with 668 participants, more than 90 percent of them believed people learn better if they are taught in their predominant learning style.” In higher education, too many of us cling to other people’s models, because we have rarely been taught, encouraged, or given the support\u00a0we need to create our own<\/a>.<\/p><\/blockquote>\n
\nwell-conceived, well-designed, well-presented courses and programs.\u201d<\/p><\/blockquote>\n
\npoint of being inscrutable, which ultimately helps QM sell an annotated\u00a0version of the rubric, as well as courses, workshops, certification\u00a0programs, subscriptions, and institutional memberships. In the rubric\u00a0itself, the words “clear” or “clarity”
\nappear over a dozen times. (There are 42 items on the latest higher\u00a0education rubric and 43 on the latest K-12 rubric, all with point values\u00a0assigned to them, a mechanism for tallying a final score.) Item 5.4 on\u00a0the higher education rubric, for example, says that “the\u00a0requirements for learner interaction are clearly stated.” While\u00a0I do think teachers should make their requirements for a course clear,\u00a0writing longer syllabi, adding more items or levels to a rubric, and\u00a0spelling out more requirements do not (in my experience) make anything\u00a0more clear for students.<\/p><\/blockquote>\n
\nin which “diversity” is mentioned). This just doesn’t go\u00a0even nearly far enough. The word “privacy” appears once,\u00a0but the onus is put on students to protect their data and privacy; the\u00a0course needs only to provide the necessary “information.”\u00a0Here are a few words that do not appear anywhere on the Quality Matters\u00a0rubric: “community,” “agency,\u201d “inclusivity,” “flexibility,” “joy,” “compassion,” “question,\u201d and “human.”<\/p><\/blockquote>\n
\nbefore the pandemic: disabled students, chronically ill students,\u00a0homeless students, BIPOC students, LGBTQ students, etc. We need to build\u00a0courses, and imagine new ways forward, for these students, the ones\u00a0already struggling, already facing exclusion.<\/p>\n