In #edu307, an online, writing intensive class I teach, I have to measure student dispositions, future educators.. We are using an assessment provided by TaskStream.
It is a multiple domain assessment, with each domain consisting of a single item that we then rate on an ordinal scale...to which accreditation boards then treat as numerical data.
That being said the item prompts in the disposition assessment wouldn't be bad (not sure if they are proprietary so not sharing) to drive reflective inquiry. As an online class every student builds out their website and we create a distributed network of blogs. So students could and should reflect on the expected patterns of behaviors in teachers.
Measuring Dispositions in Online Classrooms
Online teaching also provides an important avenue for measuring teaching dispositions because as a modality the pedagogy of teaching with the web exists in every class. It is just a matter of degree of how much takes place online. No class is taught just face to face anymore.
Blogging in Writing Intensive Classroom
- Platform to design a professional identity
- Reflection as a praxis
- Multimodal and digital texts reflective of today's classroom.
Measuring dispositions in an online writing intensive class do bias towards tech skills. I am not bothered, in fact, this is a perspective I want to encourage. Plus while I do believe (and we found in online reading) that are are significant statistical differences in tech skills based that would be explained by variance in socioeconomic skills we are still talking about a range of skills closer to zero.
The lack of technology skills in our K8 teachers is at crisis level if we hope for #CS4All to ever become a reality, and it isn't the tech skills I am truly after I want the inquisitive teacher who knows what they don't know and has built up the networks to go find the knowledge they need.
Still given our goals of diversifying a teaching force and recognizing the needs of my online students we have to scaffold this understanding. So I provide getting started video tutorials, a chatroom for help built into the class website, and in person and online studio time.
I have spent much of my career trying to research ways to measure dispositions. From Factor analysis of surveys, case studies, to almost ethnographies. Observation over time works best to measure but reflection works best for growth. Blogging provides avenues for both.
Harnessing Microformats to Measure Dispositions
I have written previously that HTML and microformat->JSON parsers could provide data to trigger tutorial bots. If a student has only one paragraph tag or maybe they never use headers after a certain date, etc. Yet I also think microformats would provide an easy way to collect user data in ways the learner controls how their information gets displayed.
A note on privacy
First in terms of learning analytics and student data I would rather default to students controlling the data rather than handing over the keys to the University. I want to build sustainable teaching networks over time. Shutting down the LMS doesn't allow this to happen.
I want my students to own their data and control how it is used. We are required to collect student data for CAEP. Teacher candidates poked and prodded like lab rats now. This does not mean students must cede control to some out of state board that swoops down to suck up data.
Hypothetically we can still meet both requirements of data control and assessment. By using microformats hosted on websites students control they can simply delete that source from the web. While the school may keep an archived version available, and anyone may have archived an open website the source gets removed from any record sent to outside agencies.
My Current Workflow
I have feeds for all my classes in inoreader. As a post comes in I cut and paste it into a spreadsheet with the assignment linked on top. I cut and paste each links.
With microformats this would not be necessary. In my graduate edtech classes students send reply posts to the assignment page so I can organize the webmentions I receive. A really cool teacher dashboard could easily be built eliminating my spreadsheet step.
The way our dispositions work, you are supposed to make note of student behavior such as getting an email written in an unprofessional manner or if students go above and beyond in volunteer work. So what I do is leave comments on the spreadsheet or publish private notes to myself. This could be automated using microformats.
I could highlight student work using fragmentions and then tag that to the specific disposition item adding a note as to why that evidence meets the criteria and action steps the student should take for growth.
Microformats and Machine Learning
For brevity's sake, I am going to lump automation and machine learning, disparate things, into one ontological basket. My bad.
I think microformats also provide a path to machine learning in the measurement of teacher dispositions. In my class, for example, looking at date and time of submission versus date and time assignments were due and then using this data.
The ability to analyze student work to measure dispositions will require good mark up going in. That is why I believe microformats, as a minimally viable format because it data class carried in HTML, provides the best pathway for data interoperability for blogging based classrooms.
How Would It Work?
That's the best part, microformats is probably the most widely deployed (discounting commercial search engines that need to have zero influence on my classroom) forms of metadata used on blogs. So many toolsets exist that instructional designers, teachers, and developers can use.
So I would have a blogging platform with proper microformats. This would then be parsed into JSON and can be exported to xAPI, the data interoperability standard replacing SCORM.
An HTML first approach, that microformats provides offers the easiest way to measure teachers dispositons while building critical online teaching and digital literacies skills.
In fact bit by bit on the weekends I have started to revise the UI for the rubrics we use in ReVIEW Talent Feedback System. Well my goal is twofold: Move to CSS Grid from Bootstrap and add nested h-reviews
We already export our data in really pretty JSON files that I love to run statistical tests on but the addition of h-review rubrics plus OAuth will allow us to connect to all of the tools on the #IndieWeb and any blogging platform that plays nice with microformats.
Basically, we could become the rubric system of open pedagogy. We are still going to be a platform to improve teaching through better feedback. This will include tracking dispositions.
I believe if you teach with personal websites and blogs (and you should) then microformats may provide us the best avenue forward to harness data and keep learners in control.