TEP Blog

I have been thinking lately about Assessment Design. Most likely because as I write this it’s finals time and, walking through campus, I hear students agonizing over tests, culminating projects, term papers and all manner of academic gauntlets. They are under-slept and over-caffeinated, grumbling or speaking a million miles a minute about what they have studied, what they have yet to study, what projects they have completed or are working on, and— this is what I find most remarkable— how surprised or bewildered they were by their testing. I remember this experience quite clearly. As an undergraduate there were tests that I prepared for by indiscriminately cramming as much information presented in the course as I could: names, times, dates, numbers, equations, cities and measurements, most of which were irretrievable by the end of finals week. The tests themselves felt like minefields in which I was blindsided by questions that I couldn’t connect to the course as I understood it. My internal dialogue went something like this “Okay I remember that one, okay … darn I could answer that if I could remember that woman’s name …. okay… okay …… WHAT? We we’re supposed to know that? We never talked about that? That’s not in the book! That wasn’t in any of the power point slides! When did we learn THAT?”

Now that I’ve taught courses of my own, I understand what a terrific miscommunication this was. As a teacher, I intended tests and other culminating projects to measure and assess my students’ understanding of, and ability to apply, the concepts we had worked with in class. But as a student I thought that these exercises were meant to measure how much I remembered and how quickly and accurately I could recall what I’d been told. And I reasoned, implicitly at least, that if the test to determine the success of the course was about remembering, that must be the purpose of the class. In other words, I thought that to learn something was to hear it and remember it. Now that I’m an instructor, I cringe to think this is my students’ understanding of what constitutes success in my courses. But if the ability to remember a slew of names dates and formulas, or the ability to recall information quickly and under time constraints, is the key to performing well on my tests, then that’s exactly the message I’m sending.

We all know about factors that create “noise” in our assessment data-set, like students’ varying abilities to memorize, compute sans calculator, parse the complex syntax of multiple-choice questions, think effectively under time constraints or stay focused in harshly lit and uncomfortable rooms. There are situations and disciplines in which these kinds of skills might be necessary, and are therefore legitimate targets of assessment. For example, it’s appropriate to ask medical students to memorize the major organs and their function, or to be able to diagnose a heart attack in a timely fashion. In other situations, assessment practices which rely on these skills can be inaccurate, unfair and, if my undergraduate experience is any indication, misleading. Still, instructors, myself included, continue to use these mechanisms to measure student success for various–sometimes very good– reasons. You might want to have a leisurely fireside conversation with each of your 400 students to determine if they had progressed in the course, but unless you are alloted a sizable legion of GTFs, it’s difficult to avoid Scantron entirely. So there are always limitations, always a margin for error.

At the same time, there are compelling arguments for working to reduce this margin of error— in the same way as we might work to reduce the margin of error in our own scholarly research. Because, isn’t assessment just a term long study in the efficacy of your instructional methods and the extent of student engagement? Why shouldn’t we approach our teaching with the same methodology we apply to any important issue, question or problem in our discipline?

I wonder how far I’d have gotten in my academic career if every time I encountered a writer I didn’t understand, I assumed they were a bad writer, or that they had not tried hard enough, or if the thesis of my final project was something like “Derrida really needs to rethink his sentence construction”? How far any physicist would get if, when their data patterns were different from what they had expected, they tossed the results. Or how effective an experiment would have come from a question like ” Does neon catalyze reactions?” But so often, because we are tired, frustrated, overworked and overwhelmed, we don’t have the energy to approach course design with the curiosity and creativity we dedicate to our research. Which is why, despite my best intentions, I have ended up writing comments on student papers like ” I don’t understand this” or writing a rubric that describes an “A” thesis using language that describes their work in relation to some unspecified ideal. Words like, “excellent” , well written, highly developed” which, without further information about which parts of the text are ” excellent” or what qualities and component make them “well- developed”, is neither tremendously descriptive nor, as you might guess, useful in justifying the grade should a dispute arise.

This is not to say that everything that happens to an undergraduate is the direct result of our teaching. Sometimes they just stayed up too late playing Halo and didn’t start studying until three in the morning. But it might be useful to begin with the assumption that their work contains information about their learning process, and if we are willing to critically analyze the data, we might discover how best to help them. After all, if we ask our students to be specific, accurate and critical aren’t we accountable for applying the same standards to our teaching.

Approaching teaching as research is also an excellent way to teach by example. Many of the skills we want students to learn are founded on rigorous critical thought. So shouldn’t we apply the same skills and thought process that we are teaching to methods of our assessment? In my own classes, I try my best to do this when I comment on student writing. I wouldn’t want them to use slang in their paper because it would be inappropriate and difficult to understand. For the same reasons, I don’t write comments heavy with pedagogical jargon. I ask them to make arguments based on common assumptions and understandings, arguments that work within the constraints of informal logic, arguments that utilize examples and specific references to the texts we’ve read. And in my responses to their papers, I try to make my comments clear, logical, and specific to the text. I try to make the best argument I can for my evaluation of their argument.

Though this particular example is best suited to the humanities, the principle is applicable to almost any discipline. For example, if you are in the sciences, you can use the scientific method to design your assessment. (Proponents of the scientific method, from what I’ve heard, are big on clarifying questions and eliminating extraneous variables from measurement mechanisms.) You might approach multiple-choice questions, or final projects in the same way you might devise the most efficient and accurate way to measure pheromone levels in zebra fish or the radioactivity of a waste material. Consider also that when you talk to your students about your process around assessment, you demonstrate how the skills required to practice your discipline ( critical thinking, good communication, reading comprehension, experiment design, etc.) can be very useful for answering questions and solving problems outside the confines of the course topic.

If you’d like more information on approaching your own teaching as research you can
• Contact TEP
• Go to the Brand Spankin’ New “Assessment Design” wing of our website, which will guide you through designing effective and relatively pain free assessment strategies.
• Sign up for Georgeanne Cooper’s series on “Course Design By Objectives”
• Come to my workshop on “Writing Rubrics for Difficult to Define Skills”

No Comments

Hooray for the Blogosphere!

You can see our posting here

No Comments

Recently, I participated in some Web seminars and hosted a workshop on the topic of Virtual Worlds within education. From these experiences. I have been thinking about the fascinating divisions that “seem” to occur around digital technologies. These thoughts have challenged some of my overall ideas about Web 2.0 technologies in the classroom. Of particular interest for me here are those divisions that occur between practical application and theoretical analysis, and additionally, the sub-divisions that occur in each of those main categories.

Let me explain myself in (hopefully) short detail. Yes, I am very much aware of the subtle and not so subtle complexities found within these divisions. Writing a short piece about the concepts below does not do them justice, but I feel this could be a nice initial point for others to jump from and expand into their own practical, theoretical, and pedagogical practices.

First, is the practical application of Web 2.0 in teaching and learning. There are many avenues of discourse one can take when addressing this topic, but I want to look at the term ‘digital literacy.’ There are two ways I have seen to consider this term:

1 = Digital Literacy as a category that Gilster (1997) described as “emphasiz[ing] the importance of examining the source and weighing the information against other resources, all part of the critical approach to Net content that we use when engaged in the process of knowledge assembly” (p. 239). Or along the lines of what Jones-Kavalier and Flannigan (2006) proposed as “represent[ing] a person’s ability to perform tasks effectively in a digital environment, with ‘digital’ meaning information represented in numeric form and primarily for use by a computer” (p. 9).

2 = Digital Literacy as a form of being able to properly read the digital world presented to an individual, or group of individuals, and to relate to visual and media literacy theory. What is the narrative of this image? What is this narrative telling us about us as individuals? What about the larger cultural and societal influences on that narrative? These questions for me came out of Barbatsis’ (2005) presentation of narrative theory and the need for applying traditional narrative logic to a pictorial (visual) logic. One does not replace the other; they actually build upon each other.

So we have a division created here of looking at skills for writing and reading digital media, and neither is a right or wrong approach, they are just different. That said I will note that most of the literature about digital literacy focuses heavily on the above listed #1 skill sets. In summary an important question becomes: Are you as an instructor teaching the writing or reading skills (or both)?

Second, is the interesting theoretical discourse about the digital world. What I’m thinking about here, and bear with me for I think this comes around to the previous discussion about practical applications for education, is the good and bad of hypermedia. The good has stated that we are evolving into a world that includes more access to: news and opinion, other people for social and professional networking, and lastly in terms of Web 2.0 tools for us to individually create our own New Media content. An example can be seen in the specific aspect of hyperlinking as creating a “collective intelligence”:

Hyperlinking is the foundation of the web. As users add new content, and new sites, it is bound in to the structure of the web by other users discovering the content and linking to it. Much as synapses form in the brain, with associations becoming stronger through repetition or intensity, the web of connections grows organically as an output of the collective activity of all web users. (O’Reilly)

This is in contrast to the post-modern concern of the hypermediated environment where the digital leads to a linear and binary world, in which narratives are constructed in a cold and disembodied metanarrative. As Baudrillard (1983) argued, “Digitality is with us. It is that which haunts all the messages, all the signs of our societies. The most concrete form you see it in is that of the test, of the question/answer, of the stimulus/response” (p. 115.). Current New Media theory has now turned to the concept of ‘hypermediacy’ (’remediation’), which states that New Media builds on previous media, and there is “the desire for immediacy lead[ing] digital media to borrow avidly from each other as well as from their analog predecessors such as film, television, and photography” (Bolter & Grusin, p. 9). Now the narratives are not cold and disengaged from the human experience, but are dynamic and flowing out of human traditions.

Okay, yes this is all simplified, but it does get at the more general concerns educators have about the use of Web 2.0 digital technologies in the classroom. What happens when you allow the students to become involved in a platform that is set up specifically for user control over the content? Can we avoid the coldness of pure content delivery, but still engage in a high level of scholarship? In a presentation I attended online by Aaron Delwiche (Trinty University), I was especially struck by the following last two “Tips for teaching with virtual worlds”:

5. Walk a fine line between freedom and control.
6. Anticipate blurred boundaries.

This connects nicely with the pedagogical components of creating a learning community and the +/-’s of giving up control of one’s instructional content to a certain degree. I think this is where the Chickering and Gamson (1987) “Seven Principles of Good Practice in Undergraduate Education” becomes a nice initial guiding point.

Good practice in undergraduate education:

1. encourages contact between students and faculty,
2. develops reciprocity and cooperation among students,
3. encourages active learning,
4. gives prompt feedback,
5. emphasizes time on task,
6. communicates high expectations, and
7. respects diverse talents and ways of learning (p. 1).

Consider how the above fits constructively into Web 2.0 applications, such as blogs, wikis, and virtual worlds. One can have control while still giving over some responsibilities to the students.

So, are the post-modernists or the hypermediacy theorists correct? For the time being I’ll leave that debate out there for others to attend to. But at the very least I think this bit of technology is growing and becoming part of our everyday lives. With any technology we introduce into the classroom there is a chance for problems as well as benefits. Much the same as using technology in our everyday lives, we need to be comfortable with its use in the classroom. As part of this process for becoming comfortable I encourage everyone to look into the many resources out there that support our teaching and learning. As noted previously, I have personally been challenged and moved to think more about my own position(s) within the Instructional Technology discipline.

In conclusion here are some resources that I feel may be of interest:

-The Teaching, Learning, and Technology Group (TLT Group) offers online resources and also a Friday Live Web seminar series that is free to attend: http://www.tltgroup.org/
-EDUCAUSE Learning Initiative also offer online resources and Web seminars: http://www.educause.edu/eli

-Adobe eLearning Solutions is a nice resource for eLearning examples, resources, and Web seminars (even if the focus is limited to Adobe’s products): http://www.adobe.com/resources/elearning/

References

Barbatsis, G. (2005). Narrative Theory. In K. Smith, S. Moriarty, G. Barbatsis, & K. Kenney (Eds.), Handbook of visual communication: Theory, methods, and media (pp. 329-349). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.

Baudrillard, J. (1983). Simulations. New York: Semiotext(e).

Bolter, J.D., & Grusin, R. (2000). Remediation: Understanding new media. Cambridge, MA: The MIT Press.

Chickering, A.W., & Gamson, Z.F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3-7.

Delwiche, A. (2008, February 19). Powerful but not a panacea – virtuaworlds as tools for situated learning. Presentation to ELI Web Seminar. Slide 46.

Gilster, P. (1997). Digital literacy. New York: John Wiley & Sons, Inc.

Jones-Kavalier, B.R., & Flannigan, S. L. (2006). Connecting the digital dots: Literacy of the 21st century. EDUCAUSE Quarterly, 29(2), 8 – 10.

O’Reilly, T. (2005, September 30). What is web 2.0: Design patterns and business models for the next generation of software. Retrieved February 22, 2008, from http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html

No Comments

Like most, if not all of you, I let my fingers do the walking pretty much all day at work and at home.  I check out the latest Mac News, try to maintain my connections to the Midwest, plan trips to places I’ll visit if I win the lottery, and see just how many hits my Reader’s Digest quote from 1995 gets these days (709 as I write this).

This ability to seek out information of all kinds is what I find most amazing about the net. For those of us who teach or work with teachers, it provides us with some of the best thinking out there on a variety of professional topics.  Here are a few of my favorite sites that I seem to return to again and again and again.

This list is not exhaustive (although you may be exhausted after looking at them all) so send us your favorite sites related to teaching in the comments section below.

To get started, I always refer instructors to the TEP site for great information on all manner of teaching topics.  Sure I have to say this but we get kudos from around the country on our website AND it’s really pretty.  Most of the chapters of Tools for Teaching by Barbara Gross Davis, University of California, Berkeley are available on-line.  It’s one of my absolute favorite teaching books.  For ideas on how to begin a class, you can’t beat 101 Things You Can Do the First Three Weeks of Class by Joyce Povlacs Lunde for the Teaching and Learning Center at the University of Nebraska and be sure to consider including the University of Oregon–Affirmation of Community Standards in your syllabus and discussing it at your first class meeting.   Active Learning in the College Classroom by Donald R. Paulson & Jennifer L. Faust from California State University-Los Angeles provides specific suggestions for using an active learning approach in your teaching, and Active and Cooperative Learning by Richard Felder from North Carolina State University receives a lot of notice when this topic is discussed.

Facilitating difficult class discussions is always a challenge. Luckily there are a number of wonderful resources available for instructors.  The Publications and Links section of the University of Michigan’s Center for Research on Learning and Teaching has a series of papers entitled “guidelines for discussion…” which offer great suggestions.  Lee Warren’s Managing Hot Moments in the Classroom is always mentioned on lists of must-read articles on this topic and Sensitive Topics in the Classroom and Teaching in Difficult Times and in Times of National or International Stress from the Office of Educational Development, University of California, Berkeley, are both very helpful reads.

There are a wealth of resources on the web that can help instructors create more inclusive classrooms.  TEP’s A Framework for Creating Success for All Students is a great place to being.  I helped develop this so I may be biased when I say “it’s terrific!”   Creating Inclusive College Classrooms by Shari Saunders and Diana Kardia, Center for Research on Learning and Teaching at the University of Michigan, is a classic article with specific suggestions.  Another must-read piece is Teaching in Racially Diverse College Classrooms from the Derek Bok Center for Teaching and Learning, Harvard University.  Finally, if you want a series of bulleted suggestions, check out Teaching, Technology, and Diversity from the School of Education, Indiana University-Bloomington.

On a more general note, I visit The Chronicle of Higher Education and Inside Higher Ed every morning when I first get to work.  It’s good to know other schools are having issues and that we’re doing better than some and not as good as other institutions.  I also recommend Diverse Issues in Higher Education and believe everyone on the campus should at least skim The Oregon Daily Emerald—either online or pick up a hard copy. 

For a more whimsical look at higher ed, Teachable Moments from Inside Higher Ed is good and the annual Beloit College Mindset List reminds me on an annual basis that I’m not as young as I think I am.

Finally, consider subscribing to the Tomorrow’s Professor Listserv,  “Desk-top Faculty Development, One Hundred Times a Year” from the Stanford Center for Teaching and Learning.

Oh, and the Reader’s Digest quote?  “Yes, risk-taking is inherently failure prone, otherwise it would be called sure-thing taking.”

No Comments

Welcome to the TEP Blog!

If you’ve made it this far, you either know what a blog is and are wondering how it might serve something like university teaching (in which case you can scoot forward a few paragraphs) or you were curious enough to check out this newish platform. So …. what is a Blog?

The word is a portmanteau of “web log.” Blogging software and services allow anyone with a pinch of computer savvy to create and maintain a web page, the main feature of which is a text entry accompanied by pictures, short video or audio clips. Often words within the text are hyper-linked to other relevant pages. Hyperlinks allow readers to customize their text, linking when they need an explanation, definition or further context. ( Try clicking on the highlighted word above.) These entries are displayed in reverse chronological order, so the most recent entry is what you see first. Many blogs allow reader commentary, tagging ( a user-generated indexing system) and RSS feeds, a subscription service that alerts readers to new blog posts. That’s what this icon denotes:

That is the “what” of blogs, as in “what the tool allows you to do.” But there is a “why” to any tool as well; the same technology can have many applications. Take, for example, the hammer, as in “teach carpentry and not the hammer.” The hammer allows you to concentrate and magnify force. But the hammer has a variety of applications only one of which is the insertion of nails.

The “why” of blogs is very diverse because, like many Web 2.0 technologies, they are highly adaptable to the needs and desires of the user. Blogs and their entries can serve as newsletters, op-eds, diaries, journals, photo archives, scrap books, “how- to”s, first drafts, etc. Many educators have classroom blogs where they post assignments, reminders, study questions, lecture notes, and reflections on news-stories relevant to the class. Classroom blogs allow teachers to easily and flexibly index course materials and postings. They allow students to easily navigate the course website and, because every blog posting has a space for comments, they provide an automatic venue for dialog with the professor.

Many more academics maintain “professional” blogs, where they comment on recent news and events related to their field. These can be valuable resources for students as well; the tone and habits of blogging (semi-formal reflections, dense with references to other work, current events and personal anecdotes) are an excellent entree to the practice of academic writing, without the intimidating specialist lexicon. Blogs document the playful, wandering, fertile and expansive discovery-stage of critical inquiry. They show students what it is to look at the world through the lens of our intellectual passions.Did your last road trip change your understanding of urban geography? Did your three-year-old niece perfectly demonstrate a classic theory of language acquisition? Does a recent City Council decision address the tragedy of the commons? Blog it! and encourage your students to read the blogs of other scholars in the field. What better way to engage students than to reveal an expert’s daily engagement with the themes and questions of the course?

And blogs, more than most other forms of popular literature, are invested in intertextuality. Every decent blog links to other authors’ writings and most blogs review or critique both primary and secondary sources. Good bloggers are in the habit of situating their own intellectual work in the context of other thinkers. What a great skill for students to see demonstrated in this relatively informal way.

There are folks reluctant to integrate blogs and many other Web 2.0 technologies. When they think of personal websites, they think of Myspace and similar exercises in flash and narcissism. I can sympathize with the concern that students might be distracted by the bells and whistles of the software or that the internet produces unreliable, unsophisticated, vanity literature of the sort students shouldn’t be mimicking. And sure, there are blogs that will rot your brain out from under you, and they may very well outnumber those blogs with relevant substance. But that doesn’t mean the genre inherently leans toward mediocrity, or that it encourages students to do so. We assign students to watch television documentaries, or make their own documentary shorts, though thoughtful examples of this genre are far outnumbered by “reality programing.”

Nor does it mean the tool will do the teaching for you. Students must learn how to use blogs as scholars would, just as they learn to read, write, speak and present as scholars. They need clear guidelines about the purpose and criteria of blog entires and commentary, they need practice and feedback. But isn’t this what we do, or should be doing, with every new exercise and activity we give our students?

Which brings me once again to “what” and “why” and the large difference between them. I find the “why” of tools, their possibilities, much more interesting and fertile than the “what.” So I’m sometimes confused when debates about educational technology focus on the latter, or when I see instructors reluctant to explore possibilities because they worry that the tools are poisonous. And this is a perennial debate in journals, at conferences, consultations and staff meetings: is the use of technology good for students? Does it increase learning? Does it rot the brain?

And it’s a debate that’s enormously difficult to settle because technology is diverse; scientific calculators and Ipods are included. And their applications are equally diverse. While calculators seem clearly educational and Ipods clearly recreational, I ignored an entire semester of AP Chemistry via TI 82 Tetris and spent hours lingering in the galleries of LACMA while listening to museum Podcasts. So, it’s hard to pin down whether the damage or benefit of any particular instance of educational technology is in the medium or the message, the vehicle or the delivered content.

But isn’t that what artistry is all about, finding the right medium for the message? Imagine an artist, trained in the use of traditional materials, the coarse horsehair brush, the thick, richly colored paint, the woven canvas. At some point in her training she comes across a set of water colors, a fine synthetic-fiber brush, and smooth, white paper. Now she has a greater range of expression. She can use these tools to paint something light, ethereal, like flowers reflected in water. The thick brush, the oil paint, the is too chunky, too real, sloppy and visceral for this particular painting. Even later in her career, she finds digital canvases she can use to make dynamic, kaleidoscopic and haunting images- images she couldn’t have made nearly as well in traditional mediums. She won’t throw out her oil paints, their watercolor and their easel, they may still have expressions that call for these other tools.

For teachers, versatility is even more important. Unlike artists, our audiences often choose us for reasons rather unrelated to our personal style. So if we want to teach effectively, to a wide range of students, the more tools at our disposal the better. This doesn’t mean we can’t specialize, or have modes in which we are more comfortable. But it suggests that there’s benefit in expanding our toolbox just as our students expand theirs.

There is nothing wrong with oil paints, big brushes, wide strokes, or smooth paper, or digital color, or the whole genre of visual art. (Nothing wrong with chalk, blue books, office hours, Blackboard, Clickers or Podcasts.) They are perfectly good tools, depending on the artist’s need and the particular project. Every new technology still has its detractors and abstainers, and that’s fine. Not all artists go digital, not all musicians will use synthesizers, not all teachers use blogs. But some do, successfully. In fact, faculty at several universities, including our own, are maintaining blogs for their classes, as well as professional blogs within their field of interest.

For further information on the use of Blogs in the Academy please see:

An Article from the Chronicle of Higher Ed

An informal article about the common pitfalls of using blogs in the classroom and pedagogical tips for making the most of student blogs.

A More Scholarly Article from the Journal of Online Learning

2 Comments