Skip to main content

Greg McVerry

Criterion vs Holistic Rubrics? #EDU407Sum19

5 min read

You all know a rubric but have you considered the insturctional design decisions behind building and using rubrics?

 

Two Types of Rubrics

Teachers use rubrics as measurement tools to capture growth or to measure the quality of an artifact. No matter what they have three parts:

  • Criteria
  • Rating Scale
  • Indicators

 

Criterion Rubric

in a Criterion Rubric you can dilenate the scales of each thing you hope top measure. In the table below you see how the criterion go down  the left. You then have some type of numerical scale. Each level of descriptor is then used to describe hwat is required to move between the perfromance scales.

 

 

 

 

Criteria

Scale 1

Scale 2

Scale 3

Criterion X

Somewhat X

X

Totally X

Criterion Y

Somewhat Y

Y

Totally Y

Criterion Z

Somewhat Z

Z

Totally Z

 

 

Criterion rubrics can also be measured on single domains or multiple Domains. Teacher evaluatiopn rubrics often have multple Domains. Each domain contains a set or Indicators and each indicator is then broken into specific criterion.

 

 

 

 

In the CT SEED Rubric there are Four Domains. Each Domain cosist of of three indicators. Each indicator has its own criterion based rubric. These are labeld as attributes. Yoru Domain score is made up holsitically from the three indicator scores, and each indicator score is chosen from a series of attributes. Thes rubrics are huge!! Pages and pages

 

 

 

In this image above you see just one of the rubrics for jusy one specfic indicator under one Domain. All images taken from https://portal.ct.gov/-/media/SDE/Evaluation-and-Support/CCTRubricForEffectiveTeaching2017.pdf?la=en

 

What Must Criterion Rubrics Have

  • score points and criteria for progressive proficiency
  • consistent language across scales
  • define each scale point
  • requires extensive training for reliability
  • distinct criterion that do not overlap

Benefits of Criterion Rubric

  • Summative data across scales
  • long term growth over years
  • Descriptors provide feedback

Risk of Criterion Rubric

  • Low reliability-most have to use a within band, meaning off by one scale isn't considered disagreement.
  • Time consuming-take a long time to score
  • Cognitive load for learners who need focused feedback

 

Criterion Rubrics in the Classroom

In the above example of a 3rd grade criterion rubric you see students are judged on six criterion and there are three scale points. Notice the langauge of the descriptors. "important vocabulary," "some vocabulary," or "copied information"

 

Notice how "copied information" does not equal "no vocabulary?" This means a scale of 1 is measuring something different than a 2 or 3 on the word choice criteria. The descriptors matter.

 

We call those changes in descriptors "key levers" you use this language when providing students feedback in written comments or during conferences.

Holistic Rubrics

Holisitic rubrics have the same multiple criterion, scales, and descriptors. The difference is the performance is scored as a whole rather than on the individual criterion.

 

 

 

 

Scale 3

  • Totally X
  • Totally Y
  • Totally Z

 

Scale 2

  • X
  • Y
  •  Z

 

Scale 1

  • Somewhat  X
  • Somewhat  Y
  • Somewhat  Z

 

 

 Benefits of Holistic Rubric

  • Holistic rubrics are less time consuming
  • Have higher reliability-raters only have to agree on utimate score
  • Emphasis placed more on what a learner did rather than did not do

Risk of Holistic Rubric

  • No feedback for students (you can still write or give feedback using descriptors)
  • When students vary widely on the criterion you can not differentiate
  • The criterion can not be weighted.

Many state writing assessments and commercial programs use holsitic rubrics. This is due to the reduction in cost of training raters and the increased reliability.

Holsitic Rubrics in the Classroom

 

Here is an example holistic rubric. You see it is on a five point scale with four criterion. Though if you look closelsy many of the criterion cast the same ontological net...meaning they measure the same thing.

Do We Need to Rubricize Everything?

In I provided you with a self checklist. I then ask you to rate yourself and reflect on your growth using this checklist.

 

I never "score" you against the rubric. Converting categorical data into numerical data has never sat well with me. Even worse when we make up some percentage from these scales and then use them to force rank students.

 

It has so little to do with growth.

 

I don't mind using rubrics for summative data and to see patterns in the aggregate or growth over time but they are useless as tools of growth for many students. Look at the criterion rubric there are 18 boxes of quality for an 8 year old to keep track of...

 

Get real.

 

Instead I like having personal conversations with students and developing TAGs-Targeted Areas of Growth. What are the one or two criterion a student should focus on when improving writing. Never try to get an 8 year old writer to adresss six different indicators of quaility at once. I don't think adult writers should undertake such an edeavor.

 

Once you meet with a student or read their work and have them develop their targeted area of growth you can then have them focus revisions on these areas. Students can highlight changes or better yet reflect on how they addressed their growth area. This in turn can reduce your grading load.

 

A 5th grade teacher with 90 essays to read is not manageable and keeping your time in check helps to ensure quaility feedback.

 

 

CLMOOC

Prev | Home | Join | ? | Next