This is the third in a series of posts about each of the teams that will be attending SCI 2015, and their projects. This one is adapted from the text of the proposal submitted by Daniel Powell.

Guiding Questions

Digital scholarship is often a deeply collaborative and networked enterprise, one which – in its various forms – involves multiple practitioners from a variety of academic contexts. This working group believes formalised evaluative structures have not kept pace with either the realities of social knowledge creation or with the numerous technological efficiencies provided by digital tools and platforms. Credit, promotion, funding, and credentialing are more complex topics than ever, yet many individuals and institutions rely on simple, outdated strictures to make judgements.

Guided by these interrelated questions, we hope to interrogate this landscape of knowledge production and mobilisation:

• How should institutions of higher learning, and individuals embedded within those institutions, value and evaluate networked knowledge production undertaken in wide-ranging collaboration using networked digital tools?
• What is the role of digital methods and platforms in developing new pedagogical practices and curricular structures that foster digital scholarship in university classrooms? How are such tools being deployed to bridge the troubling gap between bifurcated models of “teaching” and “research” as discrete activities?
• What is the role of mentored praxis in evaluating intellectual labour and student progress?
• How can we rethink traditional models of authorship and intellectual production to ensure that the work of heterogeneous teams of knowledge producers – teams that can and do include the general public, research stakeholders, students, faculty, alternative academic staff, librarians, computer IT professionals, etc – is accurately understood and valued?
• What can we do to rethink traditional models for publishing and editorial practices, graduate and undergraduate standards for evaluation, and tenure and promotion guidelines to make them more effective in networked knowledge environments?

Objectives

Overall, our objective is to link discussions of social knowledge creation (crowdsourcing, labour practices on digital projects, wiki culture, etc) with heterogeneous knowledge producers (undergraduate students, graduate research assistants, library personnel, altac staff, etc) to produce new ways of understanding how to measure and evaluate digital scholarship.

Pragmatically, we hope to use this working group to promulgate evaluative standards and guidelines for faculty, administrators, students, and staff working within and around digital scholarship. In this effort we build on the robust foundation developed by, among others, the Modern Language Association, whose efforts in various workshops, publications, and committees have begun these conversations. [For a summary of this work, see the 2011 issue of Profession, published by MLA and available for free here: http://www.mla.org/profession] The MLA Guidelines for Evaluating Work in Digital Humanities and Digital Media – themselves the culmination of several years’ work on the topic – recognises, for example, that digital scholarly practitioners “engage in collaborative work” far more often than their non-digital counterparts, but stops short of recommending specific frameworks of evaluation. Compounding the difficulties inherent in formally evaluating digital research is the collaborative involvement of students, graduate researchers, library staff, etc. The struggle for evaluators is not limited to form (archive, blog post, database, source code) but also encompasses disciplinary norms and shared authorship practices. We hope to blend these two concerns – digital scholarship as form and digital scholarship as collaborative process – to discuss and share guidelines that address credit, mentorship frameworks, and scholarly merit for digital work.

In a theoretical sense, we hope to confront the emergent body of evidence and scholarly products that indicate a qualitative change in how knowledge is produced in a networked age. By this we mean the shift from single-author, long-form prose research (especially in the humanities) to distributed models of intellectual production that intersect with social media connectivity, digitally facilitated coauthorship, and well-defined and traceable patterns of intellectual contribution. Social knowledge creation is quickly becoming a fact of life for digital scholarship, as is the reality of iterative, long term development for digital projects.

We envision this exploration and discussion proceeding along three axes:

1. A survey and examination of existing documentation and guidelines related to digital scholarship and collaborative knowledge production. Much of this can be compiled prior to SCI and brought to bear immediately on early discussions.
2. Qualitative discussion and sharing of the institutional experiences with pedagogy, digital scholarship, and professional evaluation our working group allows. In this we will leverage the diversity of voices we have brought together at SCI.
3. Synthesis and creation of holistic guidelines and documentation for moving conversations on this topic forward in multiple departments, institutions, and scholarly organisations. These are intended for practitioners of scholarly work in digital form and in large teams to use in situations of tenure, promotion, and credit apportioning.

In large part, the challenges to new systems of evaluation and credit are not technological or infrastructural per se. Instead, they are social, habituated by longstanding disciplinary norms and expectations. They are deeply embedded in administrative norms and processes, from informal expectations to the literal paperwork used within evaluative frameworks. They find expression in tenure & promotion guidelines that ignore collaborative work or frame digital scholarship as service; in evaluation frameworks like the Research Excellence Framework in the United Kingdom or the Excellence in Research for Australia which overvalue monographs in rigid point based systems that determine funding; in requirements for depositing dissertations that preclude, by definition, digital work; and so on. Our hope is that the documents and discussion that emerges from SCI on this topic can serve as an insurgency against those forces that stifle innovative research and actively separate “researchers” from other knowledge producers on digital projects.

Relevance of Working Group Participants

Our working group has come together around a single idea: that to best discuss how collaborative and cross-demographic digital scholarship should be evaluated, multiple voices and viewpoints must be represented.
To that end, our group is diverse: tenured, internationally known researchers in the digital humanities and e-literature; undergraduate students working on digital scholarly projects; advanced graduate students who have participated in large-scale collaborative projects; alternative academic staff dedicated to fostering digital scholarship at small liberal arts colleges; early career and tenured faculty devoted to digital knowledge production at teaching-intensive undergraduate institutions; and digital scholarship librarians invested in altmetrics and distributed communities of practice. We represent departments of English and Digital Humanities, large research libraries, several digital research laboratories and centres, and Andrew Mellon funded initiatives. Geographically, we represented the United States, Canada, and the United Kingdom. Together, we represent a cross-section of digital scholarly practices in contemporary academia.

Relationship to SCI 2015 theme.

Our working group proposes to directly engage with a number of the questions put forward by SCI for 2015. In response to the multiple questions asked by SCI, we see the following possibilities for exploration and dissemination:

• In exploring what it means to be an author and how to value, attribute, and reward the work of multiple contributors, we hope to push for a redefinition of the author/researcher that encompasses individuals who are often excluded from such discussions, especially in pedagogical and mentorship contexts.
• In surveying varied systems of incentives for digital scholarship, we anticipate finding an emphasis on single-author monographs and articles. The process of challenging these norms from the multiple viewpoints of our working group – teaching faculty, librarian, student, tenured researcher – will allow us to synthesise what these frameworks should be.
• In addressing the relative value of innovative digital scholarship to multiple audiences and users, we envision an opportunity to synthesise pedagogy, knowledge creation, and diverse disciplinary activities in a more holistic framework. In other words, digital scholarship forces us to reconsider the existing separation between teaching and research, as well as the ‘research’ and ‘service’ functions of university departments and organisations.

Working Group Participants

Daniel Powell (Convener) is a Marie Skłodowska-Curie Fellow in the Digital Scholarly Editing Initial Training (DiXiT) Network, a Marie Curie Action funded by the European Commission’s 7th Framework Programme for Research and Technological Development. Based at the Department of Digital Humanities at King’s College London, he researchers collaborative knowledge creation, social editing practices, and crowdsourcing. Powell is also a Doctoral Candidate in English at the University of Victoria, where he has for a number of years been affiliated with the Electronic Textual Cultures Lab (http://etcl.uvic.ca/). At both institutions, he has worked extensively on issues of graduate training and mentorship; historicising patterns of academic behaviour; systemic discussion of university development; and large-scale digital projects. He is a member of the Modern Language Association’s Committee on Information Technology, Project Manager for the Andrew W. Mellon- funded Renaissance Knowledge Network, and editor (along with Melissa Dalgleish) of Graduate Training in the 21st Century, a project within the agenda-setting #Alt-Academy collection on MediaCommons (http://mediacommons.futureofthebook.org/alt-ac/graduate-training-21st-century). Having completed an AB at a small, Southern liberal arts college in the United States, undertaken postgraduate research in Canada, and now positioned at a global leader in digital scholarship, Powell is in a position to bring broad, comparative knowledge of multiple institutions and countries to bear on discussions of evaluation, credit, collaboration, and pedagogy.

Eric Dye is a photographer, graphics designer, and journalist, as well as an undergraduate in liberal studies at Penn State, The Behrend College. He works at The Behrend Beacon as the Creative Director and Opinion Editor. He has spent over 8 years independently studying photography and is now using those skills for portraiture, photojournalism, and running a small business. Eric also regularly blogs about photography at etdphotography.com. He attended the ACP National College Journalism Convention in Fall 2012 and Spring 2015. Supported by funding from the Undergraduate Student Summer Research Fellowship at Penn State Erie, he will be conducting research in the summer of 2015 to review and analyse the history of the locomotive industry in Erie, Pennsylvania. This research will be compiled and represented through a photo essay to be disseminated through the 12th Street Project and hosted by the Penn State Digital Humanities Lab. This project seeks to provide a resource for the past and future members of the locomotive industry on behalf of the local community as well as inspire new industry enthusiasm. Dye brings an undergraduate perspective to the working group, based on involvement with the Penn State Digital Humanities Lab and independent entrepreneurial activity.

Dene Grigar is an Associate Professor and Director of the Creative Media & Digital Culture Program at Washington State University – Vancouver. Her research focuses on the creation, curation, preservation, and criticism of Electronic Literature. This research relies on a deep knowledge of media production and is expressed through traditional publications (e.g. essays, articles, chapters) but also through varied activities involving curated exhibits and multimedia design. Her work has historically found itself at the cusp of changes wrought by the evolving notions of literature and associated literary activities as they are impacted by digital media and, so, has been continuously evolving in response to technological and cultural considerations. As Director of an academic program in a new and emerging field, Grigar has had to find ways to credential faculty, demonstrate scholarly viability of collaborative research, and develop assessment documents that evaluate excellence. She brings 25 years of teaching experience in higher education to our working group.

Jacob Heil is the Andrew W. Mellon Digital Scholar for the Five Colleges of Ohio, a consortium comprised of the College of Wooster, Denison University, Kenyon College, Oberlin College, and Ohio Wesleyan University. As part of this grant-funded initiative, he works with faculty, librarians, educational technologists, and students to design and carry out digital pedagogical projects. In scale, these range from simple digital collections through to TEI-encoded editions to GIS-enabled, mobile-ready presentation of historical maps. In his previous role as the project manager for the Early Modern OCR Project (eMOP; http://emop.tamu.edu/) in Texas A&M University’s Initiative for Digital Humanities, Media, and Culture (IDHMC), Jacob coordinated the first phase of eMOP’s international, inter-institutional collaboration designed to teach machines to read early printed materials. While his formal scholarly training is in book history and early modern English drama, he has found his way to managing large-scale collaborative efforts and, with the Five Colleges, to fostering collaboration by helping to build up a culture for which digital pedagogies and scholarship are sewn into the fabric of the liberal arts campus. Currently, in addition to working through his own questions about early modern drama and print history, he is invested in thinking through the ways in which the digital cultures of small liberal arts colleges and consortia can inform those of larger, research-intensive institutions.

Aaron Mauro is Assistant Professor of Digital Humanities and English at Penn State Erie, The Behrend College. He is the director of the Penn State Digital Humanities Lab at Behrend (sites.psu.edu/psudhlab). The Lab currently oversees three research projects, including the EULA Tool, the 12th Street Project, and the Hammermill Archive. As co-chair of the Digital Media, Arts, and Technology program at Penn State Erie, he teaches regularly on diverse topics relating to digital culture, computational text analysis, and scholarly communication. His articles on U.S. literature and culture have appeared in Modern Fiction Studies, Mosaic, and Symploke among others. He has also published on issues relating to digital humanities in both Digital Studies and Digital Humanities Quarterly. Mauro will bring a unique perspective that bridges research, teaching, curriculum development, knowledge mobilisation, and collaboration with the scope and spirit of the liberal arts.

Bridget Jenkins is an English and Professional Writing major at Penn State Erie, The Behrend College, in the class of 2016. She is currently the managing editor at her college newspaper, The Behrend Beacon, a position that allowed her to travel to recently Los Angeles for the Associated Collegiate Press College Journalism Conference. In the summer of 2015, she will be conducting research on the Masonic Temple of Erie, PA. Working in conjunction with the 12th Street Project, her work will be published in a collection that aims to record the history, culture, and contemporary voices of those living in the Erie area. Jenkins’ multimedia project will include a visual and oral history of the Masonic Temple, a building which represents a prime example of early 20th century architecture and the economic and cultural prosperity that made such structures possible. She argues that this building represents an enduring link between the early 1900s and today.

Sarah Potvin is the Digital Scholarship Librarian in the Office of Scholarly Communication of the Texas A&M University Libraries, where she holds the rank of Assistant Professor. Her recent scholarly work has examined the sociotechnical infrastructure behind digital scholarship, a category that encompasses community-building and norms, evaluative structures (ranging from formal promotion & tenure guidelines to the use of bibliometrics and altmetrics as proxies), and the development of platforms and policies. This focus is reflected in her work as a founding co-editor of dh+lib (http://acrl.ala.org/dh/); her more localised involvement with the Texas A&M digital humanities working group (co-convener), Texas Digital Library Metadata community group (chair), and the Texas A&M Libraries’ digital asset management system assessment task force (chair) and digital scholarship/media promotion & tenure guidelines task force; and membership, past and present, in international program committees for the Digital Humanities, Dublin Core Metadata Initiative, and the DSpace User Group (Open Repositories) conferences (as well as an organiser of unconferences). She has been employed in research-related positions since she was 16, most often with titles like ‘research assistant,’ ‘research analyst,’ and ‘editorial assistant,’ affording a view of authorship norms across multiple disciplines and settings, within and outside of formal university structures.

Raymond G. Siemens is Canada Research Chair in Humanities Computing and Distinguished Professor in the Faculty of Humanities at the University of Victoria, in English and Computer Science. He is founding editor of the electronic scholarly journal Early Modern Literary Studies, among the first open access academic e-journals, and his publications include, among others, Blackwell’s Companion to Digital Humanities (with Susan Schreibman and John Unsworth), Blackwell’s Companion to Digital Literary Studies (with Schreibman), A Social Edition of the Devonshire MS, and Literary Studies in the Digital Age (with Kenneth Price). He directs the Implementing New Knowledge Environments project, the Digital Humanities Summer Institute and the Electronic Textual Cultures Lab, and serves as Vice President / Director of the Canadian Federation of the Humanities and Social Sciences for Research Dissemination, recently serving also as Chair of the international Alliance of Digital Humanities Organisations’ Steering Committee, the MLA Committee on Information Technology, and the MLA Committee on Scholarly Editions. Siemens brings a deep knowledge of institutional practices related to digital humanities, extensive experience with tenure & promotion practices for non-traditional scholarship, and a firsthand perspective on mentoring students in praxis-based settings.

Sharing, Dissemination, and Follow Up Activities

We are excited by the possibilities that might emerge from our Working Group discussions, and anticipate a number of concrete results from our time together:

• A web-based and public facing collection of relevant guidelines, documentation, and academic publications. This might collect documents promulgated by scholarly associations on evaluating digital scholarship (such as the MLA, AHA, and various universities and consortia); publications on practitioner experiences in these areas; examples of collaborative digital projects that have modelled evaluation practices; and working group narratives of their experiences in existing evaluation frameworks.
• A white paper exploring the issues here outlined, emphasising especially the intersection of collaborative knowledge practices, pedagogy, and non-tenure track research activity in digital forms. This would also contain our synthesised insights into the current state of play in this area.
• An appendix to the white paper consisting of well-defined guidelines and recommendations for evaluating collaborative scholarship in pedagogical contexts. This can serve as a starting point for further discussion in numerous institutional contexts.
• A glossary of existing digital scholarship platforms that are being used to create collaborative scholarship. This might include wikis, CommentPress, the Google Drive platform, etc. A special emphasis will be put on tools & platforms used in pedagogical contexts.

More generally, it is our hope that SCI participants – in our working group and in the wider institute – take our insights and documents back to their local institutions and organisations to prompt local conversations. Our document outputs and web resources can have a cascading effect, being deployed in multiple contexts for diverse purposes.

[ Photo credits: https://www.flickr.com/photos/jenlen/4087508548/in/album-72157622638584885/ used under CC license; https://www.flickr.com/photos/imuttoo/2631466945/ used under CC license. ]