Modeling naturalistic argumentation in research literatures: Representation and interaction design issues

This paper characterises key weaknesses in the ability of current digital libraries to support scholarly inquiry, and as a way to address these, proposes computational services grounded in semiformal models of the naturalistic argumentation commonly found in research literatures. It is argued that a design priority is to balance formal expressiveness with usability, making it critical to co-evolve the modelling scheme with appropriate user interfaces for argument construction and analysis. We specify the requirements for an argument modelling scheme for use by untrained researchers, describe the resulting ontology, contrasting it with other domain modelling and semantic web approaches, before discussing passive and intelligent user interfaces designed to support analysts in the construction, navigation and analysis of scholarly argument structures in a Web-based environment. p. 3

Web-mediated computer supported collaborative argumentation, for modelling the specific types of argumentation found in research literatures; p. 4

Tools are provided for interacting with structures of argument, include visualisation tools and interfaces supporting structured dialogue. p. 4

Researchers are concerned with the significance of a contribution to the literature, but no digital library can answer the obvious – but complex – questions which are fundamental to critical inquiry, and which we seek to

• instill in our students: Which publications support and challenge this document? What is the intellectual lineage of this idea? What data is there to support this specific claim or prediction? Who else is working on this problem? Has this approach been used in other fields? What logical or analogical connections have been made between these ideas? p. 5

The above questions require semantic annotation at a different level from that addressed by conventional metadata or ontologically-based markup in semantic web research, which seek to iron out inconsistency, ambiguity and incompleteness in the way resources are characterised (clearly these are undesirable if the domain is uncontentious). In contrast, principled disagreement about the significance of a contribution, conflicting perspectives, new evidence that changes the world to be p. 5

modelled, and the resulting ambiguities and inconsistencies are precisely what define a field as research; they are the objects of explicit inquiry. p. 6

In sum, there remains a gap in the researcher’s digital toolkit: tools to track (claimed) contributions in a field, and to express, analyse and contest their significance. p. 6

“Ontologies” are the term used in knowledge modelling and agent research, and increasingly within the semantic web community, to describe a specification of concepts, attributes and relationships (Gruber, 1993). p. 6

we propose a semiformal ontology for scholarly discourse, primarily for humans to communicate through as a medium for publishing and discourse (although we envisage agents as protagonists and claim-makers at some point), with the express goal of supporting multiple (often contradictory) perspectives. In this sense it is as much an ontology for principled disagreement. It still requires consensus in the sense that participants subscribe to the ontology as a reasonable language for “making and taking perspectives” (Boland and Tenkasi, 1995), but in contrast to most existing ontology applications, stakeholders need not agree at all on the structure of the field being modelled. All modelling is interpretation, but when there is meant to be consensus, the end-user community is not given the option of disputing the ontology or the way in which it has been applied. In contrast, our modelling scheme makes it explicit that every contribution can be contested. This emphasis is carried through into the language of the user interface and help information, which talks about “claims”, and makes clear that the system’s function is to serve as a medium for supporting and contesting ideas in various ways. p. 6

Our modelling scheme comprises nodes and links. Nodes may be atomic or composite at the end user’s discretion. Atomic nodes1 are expressed as short pieces of free text succinctly summarising a ‘contribution’ (at whatever granularity the researcher wishes to express this). For instance, an (optionally untyped) atomic node might simply be the name of a new algorithm that the researcher wishes to add to the network as a contribution, e.g.: PageRank. A different, typed atomic node might summarise an empirical result: <Data> Undergraduate chemistry exam performance is doubled after training on the ChemVR system. These are now objects (loosely analogous to published websites with URLs) which others can link to in their own work (but unlike the web) using a semantically typed link. p. 7

1 We refer later to nodes as “Concepts” but in explaining the data model, have found that it is more helpful to refer to them in semantic hypertext language as nodes. p. 7

an object may optionally be assigned a type (e.g. Data, Language, Theory), stored as part of the link connecting it. By storing the node type in the link, rather than binding it intrinsically to the node, the typing of nodes is made context dependent: objects may play different roles in different contexts, since researchers may disagree on the node’s type: e.g. is this Language also a Theory? Is this based on Opinion or Data? One person’s underlying Theory may be someone else’s Problem. p. 8

In addition to atomic nodes, two kinds of composite object can be used as the nodes in Claims. A Set is a group of objects (atomic nodes, Sets or Claims) declared by the user to share a common theme and enabling them to be referenced by a single named node (e.g. Constructivist Theories of Learning). Claim triples themselves can also be linked from or to other atomic nodes, Sets or Claims. This nesting allows users to build complex conceptual and argument structures. p. 8

To illustrate claim triples, consider the following: [Decision Forest Classifier] (uses/applies/is enabled by) [Decision tree learning] This uses one of the General relations uses/applies/is enabled by to assert that the Decision Forest classifier studied in the paper uses, applies or is enabled by a well known method, Decision tree learning. The latter node was introduced in a different document, so this link has a contextual role: it locates the paper near similar claims. p. 8

[Decision Forest classifier improves on C4.5 and kNN] (is inconsistent with) [SVM and kNN outperform other classifiers] This claim uses the negative, Supports/Challenges relation is inconsistent with to link one of the experimental results of this paper to a result in a third paper. In addition to its contextual role, locating the claim near other comparisons of classifiers, this claim has a rhetorical role: it contrasts pieces of evidence that make contradictory assertions. p. 9

A link between two nodes is typed with a natural language label from a discipline-specific dialect, which in turn is a member of a generic, discipline-independent class (e.g. Problem-related; Taxonomic; Causal). p. 9

Our goal is to provide a given research community with a dialect that will cover the most common claims that they make (there may well be exceptional kinds of contributions that fall outside the expressiveness of the vocabulary, but the generic Other Link is available for those situations). p. 9

Defining relations in terms of class and dialect means the same classes can be employed by research communities who speak different “dialects”, or even different languages: one can change the dialect labels of the relations, without changing the underlying relational classes. Looking at the Supports/Challenges class, refutes is clearly a stronger term than is inconsistent with. Authors would be careful in their usage, particularly of stronger relations, but clearly they also both express the notion of a negative relationship between two nodes. We therefore add the explicit notions of polarity and weight (Table 1) which are predefined and used by the system, but end-users are not asked to provide these. Combining classes, dialects, polarity and weight means we can reason at a higher level of granularity than individual relations, in delivering services by working with relations which share combinations of these properties (Section 5). p. 10

The relational classes were originally derived from a data-driven approach of modelling naturalistic argumentation as we found it in a range of research domains, including computer supported collaborative work, text categorization, literary criticism, genetics, philosophy of computing, applied ethics of technology, and film theory. Relations common to several domains were identified which we could classify in the classes shown above: Supports/Challenges, Problem Related, p. 11

Taxonomic, Causality, Similarity, and General. Interwoven with this bottom-up approach was a theoretical strand of work, which we found enabled us to critique and validate the classes we had derived. Cognitive Coherence Relations theory (described next) provides a grounding for the relational classes, and conceives relations in pairs of opposites, such as proves and refutes, where one has positive and the other negative implications. p. 12

Theoretical basis of the discourse ontology relations The discourse ontology evolved through a combination of theoretical and data-driven processes. The theory-driven approach derived from psycholinguistics and computational research on Cognitive Coherence Relations (CCR), combined with a semiotic perspective on representation which emphasises the interpretive act of modelling (Mancini and Buckingham Shum, 2001; Mancini, 2003). p. 12

According to CCR theory, discourse coherence is a cognitive phenomenon that goes beyond any linguistic expression. It depends on the interpreter’s ability to create a coherent cognitive representation of the discourse content, by establishing coherent connections between its parts. The categories of discourse connectivity are expressed in natural language by specific indicators, but these are evidence of the deeper cognitive processes that natural language is optimised to express (Sanders and Noordman, 2000). p. 12

Comprehensive sets of parameters have been proposed (Sanders, et al., 1993; Louwerse, 2001), defining a space of relational primitives by which two discourse units can be related. The basic relations are additiveness, temporality (sequentiality) and causality. Each of these is then parameterised: additiveness can be conjunctive or comparative (similarity); causality can be actual or hypothetical (conditionality); both causal and additive relations can be semantic (e.g. cause-effect) or pragmatic (e.g. argument-claim); they can have positive or negative polarity (e.g. similarity or contrast); the order of the related units can be forward (e.g. cause-effect), backward (e.g. effect-cause) or bi-directional (e.g. list). Table 2 summarises this scheme. p. 12

Grounding discourse relationships in a cognitive theory of coherence affords a number of interesting properties for building a system designed to support naturalistic argumentation. Firstly, we have used the CCR typology as a tool to verify that the main relationships are represented in the taxonomy (Mancini and Buckingham Shum, 2001). Secondly, it grounds the discourse ontology in a set of relations which Sanders et al.’s experimental evidence substantiates as having psychological reality. In principle this gives the taxonomy stability and applicability across different disciplines, media and discourse types, and empirically, we have indeed modelled a wide variety of domains (see previous section). p. 13

Thirdly, CCR makes it possible for the discourse relations we use to be resolvable back to a small number of relational primitives and their parameters, and sheds light on the relationships between them. Representationally this is elegant (while also validating CCR’s generalisability). For instance, the General relation is-about can be re-expressed as the CCR relation elaboration (whose parametrical values are: positive/pragmatic/additive). Elaboration is a relation between two discourse units (atomic or composite nodes in the data model), one of which has the rhetorical function of explaining, expanding, articulating the content of the other unit. Elaboration has a lot in common with another positive pragmatic additive relation of comparative nature, agrees-with, whose rhetorical function is reinforcing the content expressed in one discourse unit by adding up more content expressing the same perspective. The current version of our server delivers a variety of services (see Section 5), but does not yet have a CCR-representational layer implemented; CCR has served more as a theoretical reference point and analytical validation tool in the system’s development. Once such a layer was implemented, if the user was to search for all the discourse units that agree with node X, the system would know that all the discourse units that are about node X may also be of interest. At present, these relationships can be ‘hard-coded’ in, but not inferred from CCR constraints. p. 14

As a research vehicle for developing these ideas, we have implemented a client-server system called ClaiMaker which enables distributed modelling of documents in a literature, and provides a variety of services for browsing and analysing the emergent conceptual graphs. Infrastructure details are given in Li, et al. (2002) p. 14

As the ClaiMaker prototype has evolved and we have learnt more about the problems users encounter with modelling, we have prototyped different interfaces for constructing models:

• A Web-based forms interface;
• A plug-in for authors to produce concepts whilst writing in a word processor;
• An argument map sketching interface to edit and manipulate claim structures;
• A text annotation interface to view and edit candidate concepts identified in a research article. p. 15

The first version of ClaiMaker used forms with basic features such as keyboard input, text search and dropdown lists. Its aim was to allow the project team to start inputting data as quickly as possible in order to populate a test collection that could be used for designing services. It took a stepwise approach to creating claim networks: first the user had to nominate the article they were modelling, then one form allowed her to create Concepts, another could be used to assemble Sets by searching for and selecting groups of Concepts, a series of other forms allowed claims to be made by selecting pre-existing Concepts and joining them (see Figure 3). Capture was broken down into sub-processes which meant that the user needed to understand the process as a whole in order to decide which step to take next, and also had to know where in the menu system the appropriate form was located. p. 15

One approach to tool deployment is to integrate any new tasks (in this case, argument construction and submission) with existing tools. When one needs to model the arguments in one’s own, new papers, we hypothesise that claim construction might be best done as one is thinking about the conceptual structure of one’s paper, that is, during writing, to minimise the delay between the expression of the idea in conventional prose, and its formalization. As a first step we have implemented a Microsoft Word plug-in p. 16

which authors can launch direct from the Word toolbar. p. 17

The toolbar button opens a ‘semantic annotation’ form for authors to enter the major types of Concepts in a paper as they write it. These can be classified in response to some prompts: Problem? Contributions? Uses/Applies? Improves on? Contrasts/Critiques? These prompts foreground the most important relational links in the ontology for summarising an article’s contribution, in other words, ‘promoting’ them from the longer menu of relational types available in the more complex ClaiMaker forms interface (Figure 3), and turning them into questions. Once the concepts have been saved (as an XML file), the idea is that the Concepts will then be imported into ClaiMaker and used as a basis for further Claim building. p. 17

In order to overcome the problems of holding complex models in memory, the team found themselves resorting to pen and paper for sketching drafts of argument maps. Figure 5 shows the typical kind of sketch produced as one works out the structure of the literature, prior to entry in ClaiMaker. p. 17

The use of pen and paper with a software tool is a telling indicator that it is providing inadequate cognitive support for users, and it is well established that sketching is a fundamental activity in many forms of creative and conceptual representation (Goel, 1995). The sketching was mainly driven by a desire to consolidate one’s own interpretation before committing it to the knowledge base. In the terms of Green’s (1989) Cognitive Dimensions framework, the form-filling interfaces led to “premature commitment”, by requiring users to commit to a structure before they have been able to validate it more broadly in the context of the overall structure. Consequently, a concept mapping tool has been developed, called ClaiMapper (Figure 6). p. 18

ClaiMapper is a standalone tool, based on the Compendium2 visual hypertext system (Selvin and Buckingham Shum, 2002). Instead of filling in a new form for each bipartite connection, the user can simply draw links between nodes, specifying the link type when prompted. Of particular use is the hypertext facility whereby copying and pasting a node across the maps for multiple documents (whether a Concept, Set or Map) does not literally clone it in the ClaiMapper’s local database, but simply creates a new pointer to the node: the interface updates the node’s display to indicate how many argument models the node is used in, whose names the user can display and jump to (e.g. the Concept link analysis ranking p. 19

Users can search the ClaiMaker server for existing concepts matching a selected node in a map, and can import or simply drag and drop search ‘hits’ directly into ClaiMapper, creating nodes with full database metadata, ready to be reused through connection to new structures. p. 20

Based on our experiences to date, ClaiMapper has proven to be a significant advance in supporting the cognitive demands of modelling, seeing the ‘bigger picture’, more rapidly creating claim structures, and the tool can of course be used for analysis and note-taking without ever uploading the model to the server. p. 20

As we continue with our development of ClaiMapper, we need to tackle the question of how it can more actively communicate to a user what a syntactically ‘good’ model ought to look like. One possibility, described elsewhere (Buckingham Shum, et al. 2003), is to provide readers with claim-making templates for stereotypical ‘genres’ of papers in a field. p. 20

ClaimSpotter: document analysis and annotation for claim formalization and reuse p. 20

The ClaimSpotter interface tackles the “chunking” problem identified by Buckingham Shum (1996) in a cognitive analysis of the use of graphical argumentation schemes. In essence, the user is faced with deciding what should be made into a Concept/Set/Claim for linking: what granularity, how succinct or verbose should the label and detail be, and how should it be categorised (if at all)? In the context of p. 20

therefore, the use of the original text as the basis for semi-automatic assistance in formulating claims is not straightforward. p. 21

ClaimSpotter is our first step towards an active user interface with concept suggestion and identification of potentially relevant areas in the source text. There are three elements:

• Identification of the areas where the author presents and defends her argument, combined with approaches to break up the text into potential concepts;
• Provision of additional services to promote collaboration and reuse within a group of readers/annotators;
• Provision of an interface to support the capture/editing/construction of claims based on the candidate concepts which the tool has extracted. p. 21

Enhancing a document. The first step of our approach is to identify areas where authors present and defend their argument. Since authors have to defend their position and their contributions, and relate them (through support or criticism) to the positions of their peers (an account of this strategy can be found in the Create A Research Space Model - Swales, 1990), we believe that the ability to guess the role played by a sentence in this defence, using text analysis methods, provides a valuable resource in the task of interpretation, which can be seen as the task of positioning oneself with respect to the author’s assertions. p. 21

We have started to tackle this problem by using text patterns that can be consistently associated with certain kinds of assertion to identify and categorize statements that signal stages of the argument. For example, our discourse ontology has natural language labels, which can be changed to fit the dialect of the domain, so the simplest approach is to identify locations where the labels appear, or synonyms as defined in a user-editable thesaurus. This gives us an indication of where (and how) the author defends her argument. Another category of interest is statements about contributions made by the authors. These are identified using references to the document itself (e.g. “Section 2 describes…”) and references to the authors (e.g. “We have proposed…”). Once patterns such as these are combined with approaches to identify potential components of Concepts, such as noun-group identification, the system can propose a number of elements ready to use as a part of a Claim, while still leaving the reader free to edit them. p. 21

Relying on such a limited number of text patterns, although useful, does not account for the richness of expression one can use in defending one's position. In a CARS derived approach (Teufel and Moens, 2002) the role played by a sentence (e.g. introducing the authors’ work, providing background information, or supporting a cited work) is guessed from a number of annotated examples described in terms of a much more exhaustive range of features including (among many others) sentence content and position in the document. We have reimplemented a simpler version of that approach; details of the different document filters can be found in Sereno, et al. (2003). p. 22

To complement this approach, one could look at further means to enrich a document, for instance the inclusion of hyperlinks between topically coherent passages (Hearst and Plaunt, 1993) or between a term and its definition (Blustein, 2000). Figure 7 shows how candidate relations and some specific areas of a research paper are highlighted in the ClaimSpotter interface. Sereno, et al. (in press) report an empirical study into how researchers annotate a research paper informed the design of ClaimSpotter, and a formative usability evaluation study of the interface. p. 22

Promoting collaboration and reuse. The second element of our approach aims at incorporating and making use of the Claims encoded by fellow readers, and the Concepts they connect. Displaying the position defended by fellow annotators as a set of Claims indicates what has been said already about the document, including readings that are different in emphasis or focus from the author’s primary narrative and argument. Figure 8 shows the usage of a Concept over the corpus of documents. The Claims in which it has been used, and the documents which it has annotated can be accessed from there. In this way, documents become connected through common Concepts, even if they do not directly reference each other. This provides a form of extended ‘semantic co-citation’ which exploits the web of structured annotations and extends the citations of a document. p. 23

Semantic Blogging for Bibliography Management, Hewlett-Packard Research Labs: http://www.hpl.hp.com/semweb/biblio p. 24

Semantic Blogging Project, Knowledge Media Institute, Open University, UK: http://kmi.open.ac.uk/projects/semanticblog p. 25

One important formal notion is “normalisation”, that is, ensuring that there is only one entity in a model representing a particular concept. In the ScholOnto approach we have never tried to enforce normalisation. It is intended as a collaborative system with no “master view”, leaving open the possibility that if one user considers that his notion of, for instance, “ontology” is different to an existing one, there should be no restriction on him creating an identically named node: competition over the definition and ownership of terms is a natural part of research, and not a practice which we could or should suppress. However, we do have some mechanisms for avoiding unintended duplication of nodes which would impair the usability of models. ClaimSpotter detects existing node labels and highlights them where they occur in the text of the document being analysed. Users of ClaimSpotter appreciate this feature because it saves them the work of creating new nodes which they wish to reuse, presenting the information proactively in contrast to a time-consuming database search on potentially unknown keywords. p. 26

One user who tried both ClaiMapper and ClaimSpotter observed that the latter led her to focus on concepts while the former encouraged the building of webs of relations. This seems a reasonable remark on the basic affordances of the two systems. ClaimSpotter takes a text and highlights interesting chunks. Chunks of text look like concepts, leading users to naturally think of that facet of the process. In contrast, ClaiMapper provides a canvas for users to lay out concepts, offering tools to organise and link them. The primacy given to a physical representation of the network may encourage users to craft an interconnected p. 26

model. Other users of ClaimSpotter commented that they would have liked to have such a visualisation of the claims they were building (which has led to subsequent work to generate graph structures from the individual claims they construct using the traditional web form interface). p. 27

These early observations lead us to believe that the design of interfaces for creating claim networks, and possibly argument models in general, may influence the kinds and quality of models produced. Comparative studies are needed to analyse whether different interfaces bias users to produce different styles of model. However, we have found that users benefit from automatic support of the modelling process, particularly through text analysis. When support encourages good practice, for example, by highlighting existing concepts and making it easy to reuse them, users welcome this and take advantage of the functionality. This indicates that if other kinds of support were incorporated into an interface, such as suggesting appropriate link types for concepts of a particular type, users would find this valuable ‘scaffolding’ as they sought to build rigorous, elegant models. While the formality/usability balance may be hard to define, good interfaces which provide users with support for key aspects of a representation, can allow untrained users to push the balance towards formality. p. 27

In previous papers we have detailed a variety of mechanisms for delivering computational services over the conceptual graph of claims that is built as researchers submit their annotation models to the ClaiMaker server (Buckingham Shum, et al., 2002; 2003; Li, et al., 2002; Uren, et al., 2003a; 2004). We summarise these here to convey the end-user’s interactional experience, as enabled by the underlying discourse ontology, and refer the reader to the above papers for implementation details. p. 27

‘Discovery Services’ that users can access fall into two broad classes:

• Graph theoretic analysis of claims networks by exploring the topography of networks
• Example: Cluster Analysis identifying dense networks of concepts suggesting a coherent topic
• Semantic analysis of claims networks which exploits the relational types p. 27

• Examples: Perspective Analysis which generates a report of supporting or challenging papers; Lineage which traces the work on which the current paper directly builds, and its converse, Descendants (i.e. measures of semantic impact, including but going beyond citations). p. 28

ClaimFinder, which delivers the services as tabs on a web page, rather than as items embedded in a drop-down menu in ClaiMaker. The default page provides a simple, single-field form for users to do keyword searching, with ‘advanced’ search tabs delivering encapsulated services such as Perspective Analysis and Lineage (Figures 10a-c). p. 28

On invoking one of the above ClaimFinder services, instead of returning a list of results, the tool generates interactive visualizations (currently in two possible formats) of the argumentative claim structures in which the relevant Concepts/Sets/Claims are embedded (Figures 11-13). These can be browsed by selecting a node to see the underlying detail, the source document it originates from, or to reveal/hide structure by zooming, rotating or filtering the number of links from the selected node. The visualization tool illustrated in Figure 12 is delivered via a Java applet when generated in response to a query, but it is also available as a self-contained Java application. The advantage of this is that if the user wants to save an argument map layout for future use, the application version can be used to open and display it, off-line if necessary. We anticipate that this will be particularly useful when crafting map layouts as instructional aids or ‘portal maps’ for students and research peers.5 p. 28

Screen recordings with commentary illustrate the tools’ interactivity more effectively than static screens and text: http://claimaker.open.ac.uk p. 28

TouchGraph: www.touchgraph.com p. 31

Consider a common question that many researchers bring to a literature: “What arguments are there against this paper?” Despite the centrality of such a notion, there is not even a language in which to articulate such a query to a library catalogue system, because there are no indexing schemes with a model of the world of scholarly discourse. There is no way to express the basic idea that researchers disagree. If we can improve on this, then we have a good example of the argumentation taxonomy adding value over existing retrieval methods. p. 32

How can we realise such a query? First, we are looking for arguments against, which map to the taxonomy as negative relations of any type (recall that all relations have positive polarity or negative polarity). At a trivial level, this paper corresponds to the currently selected document in ClaiMaker.7 More substantively, this paper refers to the claims that researchers have made about the document, specifically, the nodes linked to it. Moreover, we can extend this to related nodes, using the following definition: the extended set of nodes linked by a positive relation to/from the document’s immediate nodes. p. 32

For the given document, this discovery service does the following:

• finds the nodes associated with that paper;
• extends the set of nodes by adding positively linked nodes from other papers;
• returns claims against this extended node set. p. 32

A common activity in research is clarifying the lineage behind an idea. Lineage is essentially ancestry and (with its inverse, the descendant) focuses on the notion that ideas build on each other. Where the paths have faded over time or been confused, uncovering unexpected or surprising lineage is of course a major scholarly contribution. We have a more modest goal to start with in ClaiMaker: to provide a tool to pick out from the “spaghetti” of claims, candidate streams of ideas that conceptually appear to be building on each other. Our lineage tool tracks back (semantically, not in time) from a node to see how it evolved, whereas the descendants tool tracks forward from a node to see what new ideas evolved from it. Since descendants are the inverse of lineage (and are implemented as its literal inverse) we will only discuss lineage. p. 33

So, let us consider a new query: Where did this idea come from? A claims network can be treated as a graph, with nodes as vertices, and the links between nodes as edges. A path in a graph is a sequence of connected edges. A lineage can be conceptualised as a path in which the links suggest development or p. 33

improvement. The problem of finding lineage in ClaiMaker can then be formulated as a path matching problem, a well known problem in graph theory for which algorithms exist.8 To provide lineage analysis as a ClaiMaker service, path queries are constructed from link-types using a set of primitives. For example, we can search for paths that may be of any length, and which contain (in any order) any of the positive links that have type similarity in either direction, or the two general links uses/applies/is enabled by or improves on, going in the direction away from the target node of the query. p. 34

The improves on link type is included to reflect the notion of progress implicit in lineage, while uses/applies/is enabled by has a weaker implication of “building upon”. In CCR terms these are both positive semantic causal relations: in the first case, one phenomenon causes its own improvement by the other in the same way that a problem calls for being given a solution; in the second case, one phenomenon is a direct cause or condition for the other to take place. p. 34

The similarity links - which constitute positive semantic comparative additive relations in CCR terms - are included because if a new node is like another that improves on a third, then the new node may well also be an improvement. Similarity links are acceptable in either direction because comparative relations are bi-directional (if A is like B, then B is like A). p. 34

Summarising, from the CCR viewpoint, the functionality of lineage needs to always follow positive relations, and they need to be either causal or comparative: either they denote a step forward along a development line, or a convergence across different lines. Figure 3 shows examples of acceptable paths that could be returned by this lineage analysis. p. 34

Research related to this work can be broadly grouped into the following categories:

• research into modelling natural argumentation;
• research into Web-based annotation;
• research into concept mapping;
• research into modelling scientific discovery. p. 37

Firstly, the research community represented by the series of workshops on Computational Models of Natural Argument (CMNA), and this special issue, is an obvious source of comparative approaches. p. 37

The emphasis in this field to date has been on the scope for computational reasoning even in the face of the informality found in natural argumentation, and we are now considering how the lessons learnt from this artificial intelligence research strand can be integrated with our own infrastructure, to add computational services when patterns can be detected in the claims networks. However, our philosophy of imposing minimal constraints on the degree to which analysts structure their work places our system at the informal end of the spectrum compared to other CMNA research. As a counterbalance, however, we note with interest that strong critics of formalization in interactive systems maintain that our approach is still too formal (Marshall and Shipman, 2003). Our efforts to negotiate the ‘formalization tightrope’ will continue, with potential benefits to be accrued both through the judicious addition of computational services, whilst remaining acutely aware of the dangers of over-structuring interaction. p. 37

The approach presented here shares some of the aims of annotation technologies. Ovsiannikov, et al. (1999) analyze common practices of traditional hand-written annotation and identify its primary uses as: “to remember, to think, to clarify and to share”. They observe that the first three are predominant for traditional annotation which, with the exception of reviewing, is a largely private affair, but that sharing becomes more important for software an-notation systems which facilitate collaborative annotation. However the decisive benefit of annotation technology over traditional annotation is searchability. This reinforces our view that developing the search interface and services of the ClaiMaker system is central to encouraging and sup-porting knowledge capture. p. 37

The TRELLIS system is a rare example of a system which adds a semantic element to annotation by linking statements drawn from web documents using a set of discourse, logical and temporal connectives (Gil and Ratnakar, 2002). p. 38

Thagard’s (1992) work on modelling scientific revolutions complements our work. Using a knowledge representation scheme focused on the conceptual structures behind competing theories, he adds parameters to provide a quantitative indication of the ‘explanatory coherence’ of a given theory, given the available evidence and competing theories. Thagard’s work contrasts with ours in its dependence on an expert modeler codifying theories at a finer granularity and with greater care than we can assume with our envisaged end-users. The target of his modelling is complementary in the sense that our discourse ontology is designed to support the collaborative construction of claims – a form of computer-supported collaborative work – in contrast to the modelling of a well-understood debate, in which it is clear whether, for instance, a hypothesis has been refuted. p. 38

The most recent work has been to complete formal user testing. Firstly, Sereno, et al. (in press) report on an evaluation study of ClaimSpotter. Secondly, an evaluation study has been conducted in which the same literature was reviewed using ClaiMapper and ClaiMaker. The resulting claims network was then studied by other researchers, using either ClaiMaker and ClaimFinder, or reading a traditional literature review article (Uren, et al., submitted). p. 39

10 References p. 40