Geoffrey Rockwell, “On the Evaluation of Digital Media Scholarship”, Profession 2011, pp.152-168

“Cathy Davidson has argued that we are entering a second phase that can be loosely connected to social media technologies, often given the Web 2.0 designation (“Humanities 2.0”).9 Blogs and now Twitter are examples of social media that have been adapted for research work in the academy. Such emergent forms are particularly hard to evaluate since they don’t resemble any traditional academic form and they are more about process and relationships than finished content. A good blogger (or team of blog- gers), however, does a great service to the community by tracking fast- moving issues, linking to new materials, and commenting on those issues. The better blogs will include short reviews, announcements, interesting interventions, and notes about timely matters like exhibits. Blogs, as I have learned, require habits of attention.

Each post might take half an hour to research and post. Posts may appear to be light and quick, but the good bloggers learn and practice their craft. In some ways running a blog is like moderating a discussion list. How often does Willard McCarty post a pro- vocative note to Humanist to promote discussion?

The work of facilitating the conversations we value in the humanities should not be dismissed as service; it can be closer to journal editing. Disciplines interested in human expression should take seriously new types of expression. What is really at issue is whether scholars should par- ticipate in experiments or take a critical or judgmental stance and only comment on, review, and theorize about the creative work of others. We have encoded in our departmental divisions views about the values and differences of academic work that separate the creative work of the artist from the critical work of the art historian, or the creative work of the writer of fiction from the theoretical work of the literary scholar who studies her or him. We aren’t entirely sure if the fine, design, and performing arts should be in the academy, as the language of most tenure and promotion documents shows. Imagine trying to get creative digital work evaluated when you aren’t in the art department.

The split between “interpretation” or “theoretical” or “analytical” work on the one hand and, on the other, “archival work” or “editing” falls apart when we consider the theoretical, interpretive choices that go into decisions about what will be digitized and how. (Davidson, “Data Mining”) In addition to the problem of assessing new media work, there is the perception that at best digital scholarship is essentially community work, editorial work, or a form of translation and therefore theoretically light. It needs to be said over and over that there is nothing a priori untheoretical about digital work; it is rather a form of potential theory. I have argued that specifications, for example, instantiate a particular theory of text, and others have argued that prototypes can reify arguments. Every decision of the TEI about how to encode some phenomenon that we take for granted, like a date, is based on a theory of what a date is for the purposes of tex- tual representation. Every research tool bears a theory about the practice of interpretation and the potential for computer-assisted interpretation. Specifications and tools can be done well and be appropriately theorized, or done poorly without a view to the fabric of humanities knowledge. If we don’t recognize and support well-theorized specifications and tools, we will have to live with those that emerge from other groups with needs and questions other than those we care about. Do we really want our tools to be built only by Google and to thus be geared for handling business documentation? Likewise, if we don’t recognize the care and work that goes into maintaining the research commons through editing, blogging, and other social research activities, then our public intellectual space will be managed by others (or simply not be there).

I will go further and say that practices of theory that do not, where appropriate, take into account their implementation are unethical, especially when consequences are openly discussed. The old way of doing theory is premised on an unexamined view that the way ideas are transmitted is primarily through chains of books by great men. This is simply no longer true, if it ever was. The epidemiology of ideas — the way ideas are transmitted, explored, refined, and forgotten — is complex and changing. The Internet is changing the ecology of transmission. A widely read blog can have measurably more readers than a published book. If what we value is appropriate intervention into the flow of conversation we call the humanities, then we need to be prepared to measure contributions, no matter what their form, in terms of their effectiveness as interventions. Counting peer-reviewed books and articles just doesn’t cut it as a measurement of impact, especially with all the problems of peer review and its particular economy. It should be noted that one relevant feature of the digital is that access to information can be logged and measured in ways that were unthinkable before. Viewing statistics are easy to gather for blogs, Web sites, tools, and hypermedia. The statistics we can gather have far more detail than the crude metric of peer-reviewed page counts. While neither page counts nor Web statistics really tell you whether information is having an effect, one can infer a lot more about readers from Google Analytics than one can from sales of a peer-reviewed book.

Things to discuss: What are the subject and audience of the blog? What is the contribution to the research community of the work? Are there statistics that show the reach and impact of the blog? What are some exemplary posts that show the research focus of the blog? Are there plans to archive the blog or to repurpose parts as publications?”

Taken from Geoffrey Rockwell’s ‘On The Evaluation of Digital Media as Scholarship’, Profession 2011, pp.152-168.