the CI community, as we seek to ‘join the dots’ representing our different conceptions. p. 2

This paper presents the rationale for treating Contested Collective Intelligence (CCI) as a significant and distinctive dimension of the broader Collective Intelligence design space for organizations. CCI is contrasted with other forms of CI, and building on research in sensemaking, and the modeling of dialogue and debate, we motivate a set of requirements for an ideal CCI platform. We then describe a social, semantic annotation tool called Cohere, which serves as our working prototype of the CCI concept, now being deployed in several communities. p. 2

1. there may not be one worldview subscribed to by all; 2. evidence can be ambiguous or of dubious reliability requiring the construction of plausible, possibly competing narratives; 3. growth in intelligence results from learning, which is socially constructed through different forms of discourse, such as dialogue and debate. p. 2

Collective Intelligence - like wisdom of the crowds? How much is automatic, data mined, and how much is intentional, collaborative? How much do machines think for us, and how much do they help us think for ourselves, or together? p. 2

This motivates a tool-design approach focused on the construction and management of connections — between data, knowledge resources (such as documents), ideas (including issues, options and arguments), and people. A social, semantic web application called Cohere is then described as a working prototype for testing these ideas. p. 2

At its most ambitious, research into Collective Intelligence (CI) seeks to develop the conceptual foundations and sociotechnical infrastructures to increase our ability to resolve complex problems by combining individual contributions. An important challenge for the research agenda is to devise appropriate theoretical dimensions so that we can position different notions of CI in relation to each other meaningfully, and different design dimensions so that we can articulate design spaces of possibilities for rendering these different forms of CI. The identification and testing of these dimensions is a long term challenge for p. 2

The dimension of unconscious—conscious may prove useful, since it corresponds with low level—higher order cognition: it is easy to leave traces of low level activity without thinking (e.g. what you view, bookmark or search p. 2

for), but if a contribution requires a degree of reflection (e.g. an annotation, blog post or document) then this could only be unconsciously part of a collective intelligence if it was made public without the author’s permission. p. 3

“Sensemaking is about such things as placement of items into frameworks, comprehending, redressing surprise, constructing meaning, interacting in pursuit of mutual understanding, and patterning.” (Weick, [23], p.6) p. 3

extensive research into the interplay of cognition and external representations (e.g. see the review by Scaife and Rogers [17]), is that the very process of externalizing thought shapes that unfolding understanding: from attempting to verbalize inchoate thoughts, to sketching diagrams, to codifying data in a structured symbol system. Weick emphasizes that sensemaking is always in a social context, and cannot be treated as a purely cognitive, information processing challenge: “The point we want to make here is that sensemaking is about plausibility, coherence, and reasonableness. Sensemaking is about accounts that are socially acceptable and credible. […] It would be nice if these accounts were also accurate. But in an equivocal, postmodern world, infused with the politics of interpretation and conflicting interests and inhabited by people with multiple shifting identities, an obsession with accuracy seems fruitless, and not of much practical help, either.” ([23] p.61) p. 3

Our interest is in scaffolding the higher order cognition associated with sensemaking around complex dilemmas such as these. Sensemaking is an applied research field already in dialogue with Human-Computer Interaction research (e.g. [19]), and which, we suggest, contributes to the emerging CI/CSCW research agenda. As we explain below, CI is in our view another lens on both the process and products of sensemaking, just as personal intelligence is both the “store” of accumulated memories, plus the capacity to respond appropriately to a situation by transforming memories and new data into plausible narratives about what how the world was, is, or might be. p. 3

our proposal is that more powerful scaffolding for thinking and discourse is required, in order to support the emergence of CI around complex organizational dilemmas. p. 3

“By sensemaking, modern researchers seem to mean something different from creativity, comprehension, curiosity, mental modeling, explanation, or situational awareness, although all these factors or phenomena can be involved in or related to sensemaking. Sensemaking is a motivated, continuous effort to understand connections (which can be among people, places, and events) in order to anticipate their trajectories and act effectively. […] A frame functions as a hypothesis about the connections among data.” p. 3

planning theorist Horst Rittel, whose characterisation in the 1970’s of “wicked problems” has continued to resonate since: Wicked and incorrigible [problems]…defy efforts to delineate their boundaries and to identify their causes, and thus to expose their problematic nature [15]. p. 3

appropriately constrained by the system’s information model; • Interpretation: Users must be able to ground dis/agreement in source documents if required, so that future readers not only become aware of peers’ interpretations, but can read the target of the support/challenge; • Situated: CCI functionality should be always ready to hand for an online user regardless of their activity. p. 4

an argumentative approach to such problems was required: First generation methods seem to start once all the truly difficult questions have been dealt with. …[Argumentative design] means that the statements are systematically challenged in order to expose them to the viewpoints of the different sides, and the structure of the process becomes one of alternating steps on the micro-level; that means the generation of solution specifications towards end statements, and subjecting them to discussion of their pros and cons. p. 4

Engelbart’s [9] 1963 definition of CI, to develop organizational capacity to augment human intellect, our “collective capability for coping with complex, urgent problems”, and in particular Dynamic Knowledge Repositories, a component of which captures key elements of the collective dialogue. Our approach defines a class of “augmentation system” to assist “argumentative design”, and other forms of workplace discourse such as modelling. p. 4

One approach is to deploy text analysis technologies to detect claims, agreement and disagreement in corpora. These are still in the research labs, but demonstrating potential. An introduction to some of the most promising work from computational linguistics and semantic web approaches in the realm of scholarly/scientific literature is in [8]. p. 4

Our approach is complementary, adopting a human- annotation paradigm rather than machine-annotation. p. 4

REQUIREMENTS FOR CCI TOOLS We have argued that it is important for stakeholders in an organization not only to see who their peers know, and what they are doing online (through the variety of social web traces that are now commonplace), but critically, staff need a sense of what peers are thinking, and on what issues they agree/disagree, if we are to support the emergence of what we are calling CCI around Rittel’s wicked problems. p. 4

Social/cognitive awareness p. 4

We propose an initial set of high-level functional requirements for the ideal CCI platform: • A CCI platform builds collective awareness of contested knowledge; • A CCI platform builds collective understanding of the nature of agreements and disagreements; • A CCI platform builds collective understanding of ways to resolve disagreements. p. 4

From a research perspective, the Cohere application (http://cohere.open.ac.uk) derives from work in issue mapping and design rationale [4, 7], computational modeling of argumentation [2] and computational modeling of scholarly discourse in literatures [8, 21]. One strand of our work has been to evolve the original gIBIS software for hypertextually modeling Rittel’s Issue-Based Information System (IBIS) model for argumentative design [5]. The second strand has been studying the work practices that p. 4

Worldview: The system should be as “ontologically open” as possible, such that users do not feel in p. 4

make such tools effective CI tools in real world meetings, both physical and virtual [6, 18]. The tool’s use in meetings requires the presence of a skilled facilitator. The challenge now is to evolve a tool with new work practices that will help asynchronous annotation and reflective discourse grow in scale without collapsing. p. 5

which can be filtered by connection-type, keywords, users and groups. What this model can scaffold, therefore, is the collaborative construction of Clip/Idea annotations, and inter- connections, by users within and between different groups, as shown in Figure 2. p. 5

Viewed through the lens of contemporary web tools, Cohere sits at the intersection of web annotation (e.g. Diigo; Sidewiki), social bookmarking (e.g. Delicious), and mindmapping (e.g. MindMeister; Bubbl), using data feeds and an API to expose content to other services. p. 5

The conceptual model supports the following key relationships. Users add to the Cohere system, in private, group or public spaces. They may annotate textual web resources with clips (highlighted text fragments of interest), which are optionally connected to one or more ideas (displayed as margin notes explaining the significance of clips). Ideas may be optionally classified by inventing or reusing a type (e.g. as a Problem, Opinion, Data, Theory, Prediction). Users may also connect ideas by inventing or reusing free-text expressions that capture the nature of the relationship, but these are also classified as broadly positive/neutral/negative, making them machine processable without language technology. The connections can be visualized as semantic maps (trees or networks), p. 5

Connecting ideas One of the main Cohere functionalities is making connections. Cohere allows to connect with and between other people ideas (and/or annotations) by creating meaningful connections between their ideas and your ideas, between their understandings and yours understanding. Each semantic connection is represented as a triple node- link-node. Each element of the triple can have a different author. This means that a user can also be the owner of a semantic connection between ideas/annotations created by two other users. p. 7

It is fair to say that IBIS is emerging as a ‘lingua franca’ for introducing relatively simple semantic structure to online deliberation. In addition to Cohere, Collaboratorium [12] and Debategraph (http://debategraph.org) are other prominent examples of the maturation of IBIS-based tools, 20 years on from gIBIS. p. 8

REFERENCES 1. Boland, R.J.J. and R.V. Tenkasi, Perspective Making and Perspective Taking in Communities of Knowing. Organization Science, 1995. 6 (4): p. 350-372. 2. Buckingham Shum, S. (2008). Cohere: Towards Web 2.0 Argumentation. 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse. IOS Press: Amsterdam. 3. Buckingham Shum, S. and Hammond, N. (1994) Argumentation-Based Design Rationale: What Use at What Cost? International Journal of Human-Computer Studies, 40 (4), 603-652. 4. Buckingham Shum, S., Maclean, A., Bellotti, V. M., and Hammond, N. V . (1997). Graphical Argumentation and Design Cognition. Human- Computer Interaction, 12 (3): p. 267-300. 5. Buckingham Shum, S., Selvin, A., Sierhuis, M., Conklin, J., Haley, C. and Nuseibeh, B. (2006a). Hypermedia Support for Argumentation-Based Rationale: 15 Years on from gIBIS and QOC. In: Rationale Management in Software Engineering (Eds.) A.H. Dutoit, R. McCall, I. Mistrik, and B. Paech. Springer-Verlag: Berlin 6. Conklin, J. (2006) Dialogue Mapping: Building Shared Understanding of Wicked Problems. Wiley & Sons: Chichester 7. Conklin, J. and M.L. Begeman (1988). gIBIS: A Hypertext Tool for Exploratory Policy Discussion. ACM Transactions on Office Information Systems, 4 (6): p. 303-331. 8. de Waard, A., Buckingham Shum, S., Carusi, A., Park, J., Samwald, M. and Sándor, Á. (2009). Hypotheses, evidence and relationships: The HypER approach for representing scientific knowledge claims. Workshop on Semantic Web Applications in Scientific Discourse, 8th International Semantic Web Conference. LNCS, Springer Verlag: Berlin, 26 Oct 2009, Washington DC. 9. Engelbart, D.C. (1963). A Conceptual Framework for the Augmentation of Man’s Intellect, in Vistas in Information Handling, P. Howerton and Weeks, Editors. 1963, Spartan Books: Washington, DC: London. p. 1-29. 10. Geyer, W., Dugan, C., DiMicco, J., Millen, D. Brownholtz, B. and Muller, M. (2008) Use and Reuse of Shared Lists as a Social Content Type. Conference on Human Factors in Computing Systems. Florence, Italy, April 5-10. ACM Press: NY 11. Klein, G., Moon, B. and Hoffman, R.F. (2006). Making sense of sensemaking 1: Alternative Perspectives. IEEE Intelligent Systems, 21(4), 70-73 12. Klein, M. and Iandoli, L. (2008). Supporting Collaborative Deliberation Using a Large-Scale Argumentation System: The MIT Collaboratorium. Directions and Implications of Advanced Computing: Conference on Online Deliberation (DIAC- 2008/OD2008). 13. Lichan, H., H. C. Ed, et al. (2008). SparTag.us: a low cost tagging system for foraging of web content. Proceedings of the working conference on Advanced visual interfaces. Napoli, Italy, ACM. 14. Malone, Thomas W., Laubacher, Robert and Dellarocas, Chrysanthos N., Harnessing Crowds: Mapping the Genome of Collective Intelligence (February 3, 2009). MIT Sloan Research Paper No. 4732-09. 15. Rittel, H.W.J., Second Generation Design Methods. Interview in: Design Methods Group 5th Anniversary Report: DMG Occasional Paper, 1972. 1: p. 5-10. Reprinted in: Developments in Design Methodology, N. Cross (Ed.), 1984, pp. 317-327, J. Wiley & Sons: Chichester 16. Russell, D.M., Stefik, M.J., Pirolli, P., & Card, S.K. (1993). The cost structure of sensemaking. Proceedings of InterCHI '93, pp. 269-276. Amsterdam: Association for Computing Machinery. 17. Scaife, M. and Rogers, Y. (1996) External cognition: how do graphical representations work? International Journal of Human-Computer Studies, 45, 185-213. 18. Selvin, A. and Buckingham Shum, S. (2009). Coherence, engagement, and usefulness as sensemaking criteria in participatory media practice. In: Sensemaking Workshop, ACM Computer-Human Interaction (CHI). Conference, 2009, 4-5 April 2009, Boston, MA, USA. 19. Sensemaking Workshops, ACM Computer-Human Interaction (CHI) Conferences 2008 and 2009 20. Tziralis, G., G. Vagenas, et al. (2009). Prediction Markets, an Emerging Web 2.0 Business Model: Towards the Competitive Intelligent Enterprise. Web 2.0: 1-21. 21. Uren, V., Buckingham Shum, S.J., Li, G. and Bachler, M. (2006). Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. Int. J. Human Computer Studies, 64 (5): p. 420-445. 22. Walton, D. (1996). Argumentation Schemes for Presumptive Reasoning (Studies in Argumentation Series), Mahwah, N.J., Lawrence Erlbaum Associates, 1996. 23. Weick, K.E., Sensemaking in Organizations. 1995, Thousand Oaks, CA: Sage Publications. p. 8

Editors. 1963, Spartan Books: Washington, DC: London. p. 1-29. 10. Geyer, W., Dugan, C., DiMicco, J., Millen, D. Brownholtz, B. and Muller, M. (2008) Use and Reuse of Shared Lists as a Social Content Type. Conference on Human Factors in Computing Systems. Florence, Italy, April 5-10. ACM Press: NY 11. Klein, G., Moon, B. and Hoffman, R.F. (2006). Making sense of sensemaking 1: Alternative Perspectives. IEEE Intelligent Systems, 21(4), 70-73 12. Klein, M. and Iandoli, L. (2008). Supporting Collaborative Deliberation Using a Large-Scale Argumentation System: The MIT Collaboratorium. Directions and Implications of Advanced Computing: Conference on Online Deliberation (DIAC- 2008/OD2008). 13. Lichan, H., H. C. Ed, et al. (2008). SparTag.us: a low cost tagging system for foraging of web content. Proceedings of the working conference on Advanced visual interfaces. Napoli, Italy, ACM. 14. Malone, Thomas W., Laubacher, Robert and Dellarocas, Chrysanthos N., Harnessing Crowds: Mapping the Genome of Collective Intelligence (February 3, 2009). MIT Sloan Research Paper No. 4732-09. 15. Rittel, H.W.J., Second Generation Design Methods. Interview in: Design Methods Group 5th Anniversary Report: DMG Occasional Paper, 1972. 1: p. 5-10. Reprinted in: Developments in Design Methodology, N. Cross (Ed.), 1984, pp. 317-327, J. Wiley & Sons: Chichester 16. Russell, D.M., Stefik, M.J., Pirolli, P., & Card, S.K. (1993). The cost structure of sensemaking. Proceedings of InterCHI '93, pp. 269-276. Amsterdam: Association for Computing Machinery. 17. Scaife, M. and Rogers, Y. (1996) External cognition: how do graphical representations work? International Journal of Human-Computer Studies, 45, 185-213. 18. Selvin, A. and Buckingham Shum, S. (2009). Coherence, engagement, and usefulness as sensemaking criteria in participatory media practice. In: Sensemaking Workshop, ACM Computer-Human Interaction (CHI). Conference, 2009, 4-5 April 2009, Boston, MA, USA. 19. Sensemaking Workshops, ACM Computer-Human Interaction (CHI) Conferences 2008 and 2009 20. Tziralis, G., G. Vagenas, et al. (2009). Prediction Markets, an Emerging Web 2.0 Business Model: Towards the Competitive Intelligent Enterprise. Web 2.0: 1-21. 21. Uren, V., Buckingham Shum, S.J., Li, G. and Bachler, M. (2006). Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. Int. J. Human Computer Studies, 64 (5): p. 420-445. 22. Walton, D. (1996). Argumentation Schemes for Presumptive Reasoning (Studies in Argumentation Series), Mahwah, N.J., Lawrence Erlbaum Associates, 1996. 23. Weick, K.E., Sensemaking in Organizations. 1995, Thousand Oaks, CA: Sage Publications. p. 9