Brought up in the session this morning was the question of how instructors should "grade" digital projects. Caroline and I (Aurora) were talking about this very subject on our way to Columbus Wednesday night. Caroline gave a presentation at C's and I gave one at C&W: both presentations focused on encouraging students to produce digital texts which did not (necessarily) privilege alphabetic textual production. In both sessions audience members seemed to be particularly interested in how we, as instructors, had "graded" our students' projects--when the texts were obviously deficient.
I consider myself to be a pretty tough instructor (don't believe me, check out my rating ^_^) and I balk at the idea that my standards "fall" when I allow students to explore digital multimodality: the idea that effort counts when it didn't before. The thing about accumulating literacy is that it takes time: I can throw down a grade on a 3 page alphabetic narrative like nobodies business, but I pause, squirm, and revisit (sometimes more than once) the grades I assign digital projects.
Grading works for instructors the same way asking students to compose, say, a narrative through video, works--it's hard and uncomfortable. And it should be!
I think, no matter what we assign, that grading, with little variation, falls somewhere in the following schema: time + difficulty of topic x literacy = product. Depending on how much importance we assign to any piece of the equation and where we, as instructors, believe we fall on the novice-expert continua should determine how we grade any piece of text.
Isn't that what DMAC is all about? Gain some expertise so we can go back and share, teach, learn, grow, create?
I guess a part of me thinks it's a little silly to dismiss student work as "effort" or "deficient" when instructors and students alike are still trying to gain and understand the very literacies
saturating our lives.
What does everyone else think about grading and the digital? Obviously, answers are as wrapped in our pedagogical stances as the materials we discuss.
Subscribe to:
Post Comments (Atom)
11 comments:
two webtexts about assessment of student-produced new media texts:
Madeleine Sorapure's Between Modes, which is about using metaphor and metonymy to assess students' projects
and
RiceBall's Reading/Remediating the Text, which is about contextual examinations of grading in individual classrooms as well as rhetorical analyses of student projects
Do other folks have other articles to point us to?
I don't have any texts to point toward, but I can share my uber-simple personal philosophy about such things.
I follow a two-prong approach. The first thing I do is make sure I anchor the "traditional" (or alphabetic writing) aspects of the assignment on the assignment sheet and that we discuss how they are being remediated to whatever digital media my students are creating. That insures that everyone-- myself and the class-- has a sense of what I'll be looking at as I grade/comment.
Then I have the class design a rubric with me, keeping the assignment sheet in mind. That way I get a sense of where the class was going/what they value, and more importantly, that rubric gives me insight into what they believe they did (as we have probably all noticed, sometimes multi-modal projects don't reflect what the student intended).
I always ask for at least a page long reflection memo with each submission as well.
It's worked so far, but I could use some pointers, too. Sometimes I still get an assignment that by the rubric deserves a C but is clearly quality work and took both thought and effort, and in those cases I tend to mysteriously grade up. :)
I want to echo what Phill is saying a little bit. Assessment, in general, is something of a research focus for me. Well, used to be anyway. I think assessing work always comes from a constructed value structure, sometimes internalized sometimes not, of the folks that make work. In a research vein, we can use tools like rubrics to help us know what good work looks and sounds like but we also use it to explain how we make sense of that work.
The key is developing a tool that a) represents what a community of practice (a classroom, a department, a community) actually values in their work and b) having that tool be usable to that community.
So, multimedia presentations should be judged according to the communities that produce them. I think usability centered design (a concept of software interface design) and assessment have a lot to say to each other something I tried to articulate in this piece I wrote for SIGDOC 07: Distributed Value System Matrix. Not really a DigiRhet friendly article (SIGDOC is a engineering organization) but it gets at some of the ideas I am mentioning here.
The Dirrrrrty version:
1) do some work as a "community"
2) figure out what you value about that work
3) figure out what other people value about it
4) make a tool that articulates what your community and other people value about it
5) see if tool works well (i.e. everyone doesn't fail, it accounts for MOST elements
6) revise tool
7) revise work
8) repeat steps 4-8 as needed
Easier said than done but I think it is way to make sure that the vernacular gets into the conversation.
This thread about grading echoes comments Truman made about the need to honestly assess what we gain and what we give up when we introduce digital media into the classroom. And, this important call picks up some comments made this morning about how we talk and think about what we are doing.
To me, it gets at two questions I'm anxious to hear more about: The first cluster of questions: when we think about what we value and that includes spending less time on traditional (e.g., textual) assignments, how do we make space for this both within our own classrooms and within the institutional structures that shape these classrooms? For example, in a nod to WAC, if we want to infuse composing issues in more classes, how do we get buy in from institutional structures to do this?
Second, as composing with digital media calls us to work in multiple modes, we draw on expertise and people from various disciplines? How do we come up with a vocabulary that can be comfortably shared?
Looking forward to hearing more comments throughout the week.
testing - my last post was typed and disappeared
Okay - so I'll try again... In response to MP's and Truman's comments - I do think we should continue to reflect on what is lost in instruction when we add the new technological instruction of multimodality into the classroom. But, as many have observed, this is a special opportunity to craft assessments that are focused especially on new media (as many are presently doing). Cheryl commented tonight on the need for structure - not only for audiences to apprehend multimodality but also for the judgment of multimodal projects - aesthetically, technically, professionally, etc. A rubric, as Phil suggests, can be a key structure. For students especially I think that a rubric helps them focus on particular aspects of production and content --as they create-- that have been deemed important by peers and other evaluators.
I know that the question of grading comes up on my own campus when discussing the issue of multimodal texts with colleagues across the curriculum, and for that reason alone I appreciate reading others' posts so that I can refine my own thinking and response to the inevitable question.
In particular, I remember thinking about coherence two years ago when I first attended DMAC and I have thought of it a lot since then. The experience of working in iMovie for the first time and letting go of my notions of textual coherence as I attempted to layer elements in a timeline forced me to rely less on words and more on the integration of images and sounds to provide some kind of coherent experience for my viewers. I recorded lots and lots of voiceovers and ended up cutting almost all of them. It was difficult to trust the images to convey the meaning I wanted, but I couldn't see any other way to hold the movie together. The simultaneity of the timeline and the vertical arrangement of elements seems to offer something very different than the linear sequence of traditional alphabetic text. I felt that compression today in viewing movie clips and trailers--so much going on at once.
"Coherence" is one criterion we understand in written text, and it remains an important criterion in multimodal composing, a concept that relies on shared conventions and expectations between composer and viewer, and viewers always bring something to the experience with them. I refer others to an issue of New Literary History , Volume 35, Number 2, Spring 2004, which is devoted to an exploration of coherence from a variety of disciplinary perspectives, including social sciences and cognitive psychology. Several of the articles emphasize the interconnectedness of elements, and this might be useful for an assessment of multimodal texts.
One last comment is to refer you to another you tube video, this one by Ira Glass on storytelling. It's his third video out of four that has to do with storytelling, and it focuses on the gap between taste and production. I just tried to pull it up and could not, so I hope it works for you if you search You Tube. He explains that we get into this line of multimodal production because we have good taste and know what we like. However, when we first attempt to create these new media texts, we become painfully aware of the gap between our own work and what we know to be good. He shares an audio clip of himself after he had already been in the business for 8 years. He is of course a fabulous success now, but it took time. The compression and complexity of multimedia texts seems inherently difficult to me, and I think we all deserve the space and time to experiment and to produce some bad things first, as do our students. Perhaps we can take advantage of that good taste and call students to participate in their own evaluation, as Phill suggests, while encouraging them to go on experimenting. Mastery may require many years beyond college.
Couldn't figure out how to edit my comment. Here's the link to the you tube video--Ira Glass on Storytelling: http://www.youtube.com/watch?v=-hidvElQ0xE. I think you tube is temporarily down right now but should be available later.
Wow! This is all really great. I'll add in my two cents. :)
As with previous commenters, I also ask my students to write a reflection about their digital/multimodal work. I've been teaching first-year composition here at OSU, and throughout we've focused on things like audience, genre, purpose, etc. So, I really push my students to consider that every choice they make has a consequence, and those choices aren't always wrapped up in "technological expertise." Some of the simplest multimodal creations can be the most effective -- if there's a purpose to it. One of things that makes instructors (or, at least me) cringe most is when students create something that has a flaming red background with neon green text and auto-starting audio clips all heaped together. If the student doesn't realize that such a creation might a) induce seizures or b) annoy the heck out of viewers... well... then he or she kind of has a problem there! And, while certainly a person's rhetorical choices are wrapped up within the limitations of a particular technology or knowledge of that technology, I do think that the excuse of "I don't know how" only goes so far. And, to be honest, many of my students are digital superstars who know far more than I do about other ways of composing. But I think that reflecting can be so very important to assessment: why you're doing what you're doing, and how a particular medium/genre/technology affords you with those choices.
Hopefully that makes sense. I'm running on very little sleep. :)
I think Jackie's comment about coherence hits the nail on the head for me. We don't want students to simply assemble discrete bits of different modalities; we want them to compose a piece that uses different modalities. The tricky part is helping them to understand how parts can work together and why they should.
Hmmmm... what would Aristotle say here?
Trish
I think Jackie's comment about coherence hits the nail on the head for me. We don't want students to simply assemble discrete bits of different modalities; we want them to compose a piece that uses different modalities. The tricky part is helping them to understand how parts can work together and why they should.
Hmmmm... what would Aristotle say here?
Trish
Post a Comment