Abstract: To thrive, the Open Educational Resource (OER) movement, or a given initiative, must make sense of a complex, changing environment. Since “sustainability” is a desirable systemic capacity that our community should display, we consider a number of principles that sharpen the concept: resilience, sensemaking and complexity. We outline how these motivate the concept of collective intelligence (CI), we give examples of what OER­CI might look like, and we describe the emerging Cohere CI platform we are developing in response to these requirements. p. 2

resilience p. 3

In an OER context, it is noteworthy that it has also established itself in the learning sciences, as a disposition reflecting perseverance when stretched during learning beyond one’s intellectual and emotional ‘comfort zone’ (Carr & Claxton, 2002; Deakin Crick, et al. 2004) or when confronted by personal and social stressors, often due to poor socio‐economic conditions (Roberts, 2009). p. 3

However, in a system concerned with higher order cognition such as a community of inquiry or an innovation network, we move from simple positive/negative feedback loops, to epistemic constructs such as ideas, questions, predictions, dilemmas and evidence, and emotional constructs such as surprise, reputation, hope and fear. In other words, feedback/self‐awareness implies the capacity to reflect, learn and act effectively, both individually and collectively — a working definition of Collective Intelligence (CI). This motivates, therefore, the proposal that good CI infrastructure (people+processes+technologies) is worth designing to advance the OER movement’s resilience. p. 3

Table 1: Principles from “resilience thinking” (Walker, 2008) and their possible implications for OER collective intelligence infrastructure p. 3

Diversity  Diversity of participants and viewpoints: design for as wide a constituency as possible; do not lock participants into any worldview; support diversity, disagreement and quality debate Modularity Support loosely coupled applications/services and linked data, enabling interoperability and mashups with diverse end‐user tools relevant to OER (e.g. Google Maps; GapMinder data visualization; YouTube movies; Wikis; Blogs).  Practical experimentation with feedback loops Improve awareness of the existence, and success/failure of OER resources or ideas  Trust/social capital  Make use of appropriate measures of social capital, authority and reputation within the community p. 3

If we elaborate the issue of feedback loops, for example, the OER design lifecycle typically ceases after “publication”. Comparatively few OERs are evaluated, and our current infrastructures have weak capacity to track and learn from what happens next. p. 3

The work of Snowden and colleagues (e.g. Kurtz & Snowden, 2003; Snowden & Boone, 2007) is one approach to bringing together sensemaking and strategic thinking, distinguishing known, knowable, complex and chaotic problem spaces (Figure 1). p. 4

Browning and Boudès (2005) provide a helpful review of the similarities and differences between Snowden’s and Wieck’s work on sensemaking, with particular emphasis on the centrality that narrative/storytelling play in their proposals for how we manage complexity. Table 2 (left column) draws on the key features they and Hegel, et al. (2010) p. 4

identify, while the right column suggests ways in which sensemaking infrastructure might be shaped in order to tackle some of the breakdowns in individual and personal sensemaking that are known to occur in complex domains. p. 5

Table 2: Sensemaking phenomena in complex domains, and the potential roles that sensemaking infrastructure can play  p. 5

Sensemaking Phenomenon in Complex Domains Dangers of entrained thinking from experts who fail to recognise a novel phenomenon Complex systems only seem to make sense retrospectively: narrative is an appropriately complex form of knowledge sharing and reflection for such domains Patterns are emergent Many small signals can build over time into a significant force/change Much of the relevant knowledge in complex emergent systems is tacit, shared through discourse, not formal codifications (Hegel, et al. 2010) What do we mean by OER­CI? Sensemaking Infrastructure Opportunity Pay particular attention to exceptions Open up to diverse perspectives Stories and coherent pathways are important Reflection and overlaying of interpretation(s) is critical In addition to top‐down, anticipated patterns, generate views bottom‐up from the data to expose unexpected phenomena Enable individuals to highlight important events and meaningful connections, which are then aggregated Scaffold the formation of significant inter‐personal, learning relationships, through which understanding can be negotiated flexibly p. 5

OER practitioners and researchers come from many intellectual traditions. What “counts” as legitimate evidence in order to make claims varies accordingly. Thus, we envisage pooling an evidence base that makes clear which of the following “evidence layers” underpin a particular OER or concept (Table 3). p. 5

Technical Reports on Design Principles: Such principles may be of value to those making an OER selection decision (e.g. the following pedagogical philosophy and disciplinary principles informed the OER design, here is the rationale behind the use of the particular multimedia presentation mode.) Contexts of Use: A description of the curricular locations where a particular OER might fit and the characteristics of the student population that would typically use the OER (e.g. this introductory course in symbolic logic is a requirement for computer science majors. Students who take the course are usually sophomores and over half of them are philosophy majors.) Anecdote: Stories perhaps using text/images/video from the field that can help build understanding, even though they may lack hard evidence or conclusions (e.g. we’ve just completed the first trial of this OER and it has not met our hopes — but we have some clues as to why, which we’re chasing up.) Comparative Review: Analytical comparisons of OER materials aimed to identify strengths and weaknesses in terms of learning resources, technical requirements, and content coverage and treatment (e.g. we have classified these OER in terms of their technical requirements and how these match to assistive and mobile technologies.) Portraits: Illustrations of OER in use similar to what Lawrence‐Lightfoot calls portraitures, that is, qualitative accounts of “the complexity, dynamics, and subtlety of human experience and organizational life” (e.g. we followed, videotaped, and questioned a user over a specific chunk of time and across multiple settings and present here some unintended side effects of simple design, sequencing, and formatting decisions.) Case study – anecdotal with informal evidence: Partial descriptions and data that would benefit from further analysis and discussion (e.g. we have the following screencasts and interview MP3s that we’re happy to share because we need help to analyze them.) Case study – structured research methodology and data analysis: Reports about a particular situation supported by analysis that draws conclusions (e.g. this article/website tracks a cohort of trainee teachers for 3 months, as they sought to apply OER, video analysis using Grounded Theory leads us to propose three key factors that influence their success.) Controlled experiment: Supported comparative studies with qualitative and/or quantitative data (e.g. 48 undergraduate chemistry students grouped by ability and cognitive style used the ChemTutor OER to complete Module X, statistical analysis combined with think­aloud protocols supports the hypothesis, based on Learning Theory Y, that higher ability students would benefit most.) Learning Analysis Studies: Provide a detailed picture of the experience that students are likely to go through, and constitute a resource for iterative design improvement (e.g. we examined the data log files and can articulate how students benefit from the different components and instructional devices that make up this OER such as explanatory text, built­in videos, animated illustrations, self­assessment, learning by doing applets, and virtual labs.) p. 6

Cohere is based on three kinds of activity, which we use to organize this overview: 1. making thinking visible 2. connecting ideas in meaningful ways 3. providing services to analyze, visualize and track ideas p. 7

In Cohere, users may annotate an OER or any other web resource directly through their browser by highlighting and adding annotations, which (if public) are immediately visible to anyone viewing that page who has installed Cohere’s sidebar (currently a Mozilla Firefox extension2). As with other web annotation tools (e.g. Diigo; Sidewiki), one can treat annotations simply as informal margin notes or clippings, but in Cohere these can also become ‘first class’ entities that represent important “ideas” (such as a major question on which a project is working) around which a whole network of ideas can grow. Customizable icons signal what kinds of contribution analysts want to make with an annotation, such as a prediction or data (Figure 2). p. 7

1 See the OLnet Project workshop on Online Deliberation: Emerging Technologies for examples of other structured deliberation tools: http://olnet.org/odet2010 p. 7

Figure 3 shows a PhD student and a Researcher annotating an OER on Rice University’s Connexions, as part of a collaborative inquiry on climate change during the COP15 conference. Any of the annotated ideas (e.g. “We cannot know the physical and ecological damage due to climate change”) can have attached to it as backing evidence any number of ‘clips’ (text fragments) lifted from any number of websites. OERs are therefore linked not only by simple tags, but by more complex epistemic relationships. p. 8

Cohere provides a way to connect these nodes with meaningful relationships. The default set (Figure 4) can be edited by users to create a connection language that suits their interests. p. 9

As these are added, the Firefox sidebar displays connections between any ideas annotated on the website (Figure 5), now enabling navigation of OERs (or any website) by following paths/networks of meaningful relationships (recall that attached to each node there may be clips lifted from many sources). p. 9

This example shows the results of analysing the online discussion on open OER issues at the Hewlett Foundation Grantees meeting (March 2009, Monterey: http://cloudworks.ac.uk/cloud/view/980). Cohere was used to analyse the online discussion with a specific annotation schema which showed that issues were organized around five topics, shown in Figure 6: Share­ability, Effectiveness, Participation, Sustainability and Scalability. p. 10

As the web of user‐generated annotations and connections grows, there is the need for tools to track patterns of specific interest, going beyond simply viewing the whole map. Users can engage in exploratory study by performing customized network searches, reducing the complexity of the graph to sets of connections of interest. In a large, multi‐ user context, users will want to monitor specific ideas, documents, people or topics without having to manually check. Agents can be set to monitor structured search results on sub‐networks (that is to say specific semantic connections, to specific network depth on a focal idea). Figure 7 shows a “report” from an agent. p. 10

Finally, we are considering how we can crowdsource input to the evidence base from different OER communities, projects and websites. One approach is through the release of widgets (e.g. Google Gadgets) which the OER community can embed in diverse platforms. A user interface storyboard is at http://cloudworks.ac.uk/cloud/view/3239 p. 10

A large scale analysis of >100 OER initiatives is currently in preparation by the OLnet Project, and will be published using Cohere. We invite the community to pool its collective intelligence to review and extend this seed next year. p. 11

References p. 12