Highlights (58%)

Can we build a semantically adaptive personal learning environment that helps people learn mathematics and that meets reasonable criteria for sustainable growth and development? This is question that applies at the interface between participatory social media and interactive, adaptive, knowledge media. p. 1

The sociopolitical factors that dene the future of education are inextricably linked up with a variety of technical and social questions that apply in the computer-mediated learning context. Personal learning environments (PLEs) take both aspects into account as they aim to exploit usage data to provide individual users with an enhanced learning experience. The PLE approach is quite new, and it remains to be seen how it can be deployed to support computer-mediated learning in a complicated classical subject like mathematics. p. 1

My project will engage these issues in the context of two well-known existing systems, PlanetMath.org and the Peer 2 Peer University. The work will combine socio-cultural analysis, technical development work, and new data-driven recommendation algorithms. Sustainability will be a key test of the ecacy of this eort. p. 1

How do people learn mathematics? What motivates people to learn in one way or another, given that it's not always easy? Here we recall Euclid's warning: there is no Royal Road to geometry. Even so, we can imagine a road to free math (Figure 1)! p. 2

Just what does free mean in this context? I think it entails understanding the ambitions of the people involved, as well as their frustrations. Learning math may not be easy, but perhaps it can be free of unnecessary frustrations. I believe that to do this, we need to foster learner self-awareness and a realistic sense of what's possible. p. 2

Personalized learning environments can help us reach these philosophical and pedagogical goals by combining learner proling with recommendations based on both individual and cohort behaviour. The process of assimilating many small contributions into resources of high quality colloquially known as crowdsourcing will be attempted at nearly every level of this work, in hope that the system's growth and development will become fully sustainable. Mathematics makes a great test case for these approaches, because its detailed, rich, and explicit semantics will help us map more transient phenomena, like learner experience, to concrete meanings (and events) in the subject domain. p. 2

by moving learning into an computermediated context, we can begin to better understand how learning occurs as a side-eect of various interactions with the subject matter (e.g. solving problems, but also writing problems, helping others, doing expository writing, and possibly, debugging code). p. 2

we can capture extra value from learner interactions to make the system itself improve. These two dierent types of positive externality or surplus value will be our guide to μάθημα. p. 2

how learning occurs as a side-effect - Bereiter talked about activities, vs. focusing on the ideas themselves p. 2

2.1 Crowdsourcing for Education p. 4

it is helpful to return to the view of personal learning environment put forward by the progenitors of the PLE concept: Rather than integrate tools within a single context, the system should focus instead on coordinating connections between the user and a wide range of services oered by organizations and other individuals. (Wilson et al., 2006) [2]. p. 4

The key idea here is that learning takes place in a complex ecosystem. Rather than building monolithic, or even modular, educational structures, in their fullest development, PLEs will support fully distributed, decentralized, perhaps even organic, learning activities. p. 4

Lawler's attention to the bottom-up nature of Wikiversity provides an important contrast to the top-down nature of systems-engineering (a quality that applies even to systems with signicant distributed aspects). p. 4

Aaron Krowne's master's thesis [5] details his development of a system for the collaborative production of digital libraries, focusing on the early development of the now-popular PlanetMath.org.1 In a subsequent collaboration with the present author, these ideas about commons-based peer production (CBPP) are connected to various historical developments in hypertext [6]. The system's usefulness in an instructional context is discussed in Milson and Krowne [7], which incidentally also includes numerous citations to earlier literature on collaborative and online learning. A recent article by Piotr Konieczny describes a complementary approach: rather than adapting a CBPP platform for classrom instruction, he talks about how classes can very usefully participate in live CBPP settings, like Wikipedia [8].2 p. 4

My sense is that whether we are talking about engineered, democratic, or distributed solutions (and we're typically always talking about a combination of all of these), what remains in very short supply is the multi-level, multi-scale adaptability that could actually change a learning ecosystem. It is perhaps especially important to overcome the learnercentric model that is so popular these days (and is still clearly felt in the quote from Wilson et al. above). Or else, putting it another way, to admit once and for all that we're all learning and that, as they say, we all make mistakes. Fallibility is the rst step towards any legitimate sense of openness and beyond that towards true hackability a quality of certain socio-technical systems, not merely of tools, denoting the degree to which users can scratch their own itches or generate and full their plans, i.e. the ease with which which a p. 4

2Cf. these slides: http://tiny.cc/wikisymteach p. 4

complaint or request is transformed into system behaviour (cf. [9], especially the Chapter 9, Why Free Software needs Free Documentation; but note that hackability is not equivalent to software freedom). It turns out that for some important theoretical reasons, mistakes are also important for understanding motivation: Preferences often can be revealed by behavior, even when they are not implemented by behavior […] preferences can better be revealed by acknowledging mistakes. Mistakes in Choice-Based Welfare Analysis, by Botond K®szegi and Matthew Rabin [10] p. 5

Interesting - embracing mistakes to support learning, and understand about user intention/motivation? Critique of user-centeredness ("the user is always right? :)" Useful link to suthers1995designing - difference between UX and learning design - sometimes the most "helpful" program design is not the most optimized for learning p. 5

ActiveMath/LeActiveMath is in many ways the closest historical project from a topic or domain point of view. LeActiveMath was a STREP project in FP6 that ran from January 2004 to December 2006, and ActiveMath-EU was a dissemination project funded through the Education, Audiovisual and Culture Executive Agency, running from January 2007 to June 2008; the ActiveMath website is still actively posting news about related projects. For simplicity, henceforward I will refer to any work directly linked to the ActiveMath family of projects by the name ActiveMath. The eld of Mathematical Knowledge Management to which ActiveMath belongs is surveyed in [11]. p. 5

ActiveMath's its creators aimed to build an open source system that would provide mathematical course material that would adapted to a learner's goals, competency-level, and preferences. The project aimed to integrate a variety of other useful tools as modular additions (e.g. a search engine, a concept mapper, an assessment tool, and several computer algebra systems). There are many similarities between ActiveMath and my currently envisioned project: interactive exercises, a learner model that keeps track of competencies and motivations, and even the general idea of exploiting mistakes to help learners [12]. Nevertheless, I think it is safe to say that despite a concerted dissemination eort, ActiveMath didn't take o e.g. as a replacement for classroom-based mathematics learning. One reason for this is quite clear: content for ActiveMath was generally supposed to be written directly in a form of XML and additionally required authors to learn a number of additional idiosyncratic systems to deliver various extra features (like pedagogy) [13]. p. 5

This XML-based language, called OMDoc [14] is alive and well and has beneted from a decade of extensions, most vitally a series of improvements to a LATEX-based markup language called sTeX which generates OMDoc as well as the traditional DVI/PDF output [15], [16]. Arguably the current version of sTeX will still be dicult for the average user, but the authoring experience is much easier than writing raw XML. Planned improvements to sTeX would add support for learning paths through reusable content, an idea that was also pursued within ActiveMath [17]. In a sense, ActiveMath and PlanetMath complement each other perfectly. In 2001, it would have perhaps been too much to ask for a single project to deal p. 5

eectively with both semantic mathematics and crowdsourcing. ActiveMath may have already been somewhat overwhelmed with its many sub-projects [18], and PlanetMath was never able to amass the wherewithal needed to go beyond its basic system. After nearly a decade, during which time both projects seem to have wound down to a considerable extent, the time may be ripe for some productive synergy at the level of underlying ideas. p. 6

2.3 The last decade has seen signicant investment in terms of eort and resources (i.e., time, people, and money) in innovating education and training. The time has come to make the bold step from small-scale innovation research and development to larger-scale and sustainable implementation and evaluation. Sustaining TEL: From Innovation to Learning and Practice, from the preface by Paul A. Kirschner and Martin Wolpers [19] Ecacy and motivation p. 6

There is a certain folk wisdom that says it is hard to study motivation, and indeed, that in some ways it matters less why a person does something than that they do it. We are nevertheless drawn to consider motivation as a facet of self-regulated learning ([20], [21]), and in another light to assume that something always motivates uptake or buy-in when it happens. We may or may not be able to know what this is or control it, but if we want people to adopt certain behaviours, it is worth giving some thought to what their motivations may be for doing so. p. 6

Frank Pajares in says [22]: Psychologist Albert Bandura theorized that the beliefs that people come to hold about their capabilities and about the outcomes of their eorts powerfully inuence not only the ways in which they will behave but also the cognitive and aective processes that accompany their functioning. To sum up it's not just thinking you can that's important, but believing you can. Make it a justied, true belief and you'll know you can, right? This begs the question: what kind of maps or models of ecacy are realistic (i.e. justied and true)? Pajares suggests we examine a class of so-called coping models, which have the feature of expressing that Missteps are inevitable, they can be overcome, and even authority gures make them. […] Eective models convey not only the success experience itself but also the information required to succeed. Even so, Bandura suggests elsewhere [23] that in certain contexts, the veracity of beliefs may be less important the fervency with which they are held: Realists may adapt well to existing realities. But those with a tenacious selfecacy are likely to change those realities. Both authors would seem to agree that (in an educational context, but this could easily generalize) Making students aware of the self-handicapping strategies they regularly use to decrease anxiety is a critical rst step in teaching them how to circumvent such strategies. p. 6

Bandura outlines four dierent types of experiences that can build a sense of personal ecacy (ideally, overcoming anxiety in a non-self-handicapping fashion, one might hope). These are: (1) mastery experiences; (2) vicarious experiences of ecacy; (3) social persuasion; (4) and interpretation of physiological and emotional states. Bandura also comments on the role of collective ecacy in regard to, e.g., technology. Modern life is increasingly regulated by complex physical technologies that most people neither understand nor believe they can do much to inuence. Pervasive dependence on the technologies that govern major aspects of life imposes dependence on specialized technicians. One is reminded of Ivan Illich's notion of disabling professions and his arguments for greater self-suciency ([24], [25]). p. 7

Perhaps the thought begins to emerge that people are not only motivated when they view themselves as eective, but that they are also motivated to become eective. And, indeed, becoming eective would appear to be one of the dening features of learning in general. p. 7

Need for control over environment - teaching people to program, scripting, modify their environment - learnt helplessness in Blackboard does little to prepare you for the future world. Is P2PU right now much better? p. 7

We cannot bear to live our lives without some sort of content that we can see as constituting a meaning. Meaninglessness is boring. […] Boredom can be understood as a discomfort which communicates that the need for meaning is not being satised. […] I believe that boredom is the result of a lack of personal meaning, and that this to a great extent is due to the fact that all objects and actions come to us fully coded, while we as the descendants of Romanticism insist on a personal meaning. The Philosophy of Boredom, by Lars Svendsen [26] p. 7

Can we build a semantically adaptive personal learning environment that helps people learn mathematics and that meets reasonable criteria for sustainable growth and development? p. 7

3.1 Development of the Research Question p. 7

Our aim is to build a lens for looking at mathematical learning behaviour in detail. There are several criteria required for success. (X1) It is essential that enough people are motivated to use the system. (X2) It is essential that at least some of them learn mathematics while using the system. (X3) The semantic adaptivity and personalization features of the system must be shown to be helpful for learners. p. 7

(X4) We will require evidence that the system will grow, not stagnate. p. 8

Thus, in a bit more detail: the idea is to gather data on user interactions with a collection of user-contributed exercises and other learning objects, linked with encyclopedia content, forums, and other supportive learning tools, possibly assembled into courses, and importantly, made hackable in the sense discussed in Section 2.1, above. Then, we will analyse this data and develop new features that better support learning. Ultimately we want to engineer the system in such a way that these eorts become self-sustaining, and, indeed so that other distinct eorts can build on ours. p. 8

4 Specication of Research Approach Building educational systems online, in the open, we hope to exploit both highintensity, high-cost contributions, and a long tail of smaller and less intensive contributions. But, like open source, crowdsourcing is no panacea. p. 8

4.1 Situation relative to contemporary theory p. 8

Nishida's idea of shared context in motion, or basho [27], as ltered through the SECI framework of Nonaka and Toyama [28], can help us move from stakeholder groups to a clearer picture of the roles of actual participants and on to a detailed understanding of the activities which support these roles (e.g. a student's activities include going to class, collaborating on a class project, building a transcript, and ultimately gaining a skill). Activity theory seems to pick up where SECI leaves o, as it provides another Nishida-like way to understand the unity of consciousness and activity ([29], p. 7). These ideas will be used on an ongoing basis to create increasingly detailed sketches of the activities we are able to support and those we would like to support. (This approach seems best suited to problems X1 and X4; see Appendix B for an overview of the approach and some general results, and Section 5.3 for a more specic application.) p. 8

Don't quite see the connection between Nonaka's SECI and activity theory yet, need to read more of both p. 8

Sensemaking is a term used to describe the activities that go into making sense of things. Associated with this broad term is both an extensive research literature on an associated step-by-step methodology or schema for doing sensemaking (cf. [30]), and also room for creative ideas within a philosophical tradition that aims to say what sense is in the rst place (cf. [31]). For my purposes, a particularly useful angle on sensemaking is Organizational Sensemaking, as discussed by Weick in [32]. I hope to use the ideas here to move back and forth between high-level and detailed views, nding ways in which various data elements t together in a coherent causal scheme (Klein et al., [33], [34]). The idea of sensemaking should be useful for understanding what happens when p. 8

things go wrong, and, symmetrically, it should also help with guring out how to precipitate events of interest. Although SECI is all about describing dynamic knowledge creation, I anticipate that sensemaking will be helpful for nding second-order dynamics, insofar as it endorses a style of thought that not only generates, uses, and elaborates frames, but that also reframes the situation as this proves useful (see Klein et al.). Here hackability is important: if people are given a genuine chance to recongure the way they do things, they may be able to envision and reach goals that they never even thought of before! There is not room in this survey to connect the topic of sensemaking to the study of heuristics, but it is worth at least mentioning one landmark book, How to Solve It [35], and the importance of further follow-up here (we note that ActiveMath has been through this territory as well [36]). (This approach seems best suited to problems X2 and X3.) p. 9

Interesting to design the research study itself so that participants are able to co-construct / modify the learning environment… you are not presenting them with what "you" believe will be a great learning environment, but part of the research success is enabling them to constructively build something - which might be a far harder task p. 9

In light of these remarks, we will aim to be to be compatible, without necessarily aiming to support or integrate every single mathematics-related activity. Compatibility here could be on any level (e.g. a shared tool, a shared license, a shared language, or a common way of thinking about things). Planetary is the name of a system that matches this prole, and around whichmysubsequentdevelopmentgoalswillcenter.3 Planetarybeganasacollaboration between myself, Catalin David, Deyan Ginev, and Michael Kohlhase at Jacobs University, with the goal of creating an easy-to-extend clone of PlanetMath's software platform Noösphere (see [37]). We chose to base the system on the popular open source Vanilla Forums4, and the rst set of extensions added mathematical writing and rendering capabilities using LaTeXML5. Subsequent extensions all pure plugins to Vanilla have begun to integrate the KWARC stack of software tools for working with mathematics with semantic markup.6 p. 9

3 http://trac.mathweb.org/planetary 4 http://vanillaforums.org 5 http://dlmf.nist.gov/LaTeXML/ 6 http://kwarc.info p. 9

My development goals (see Figure 2) are to p. 10

(K1) nish the clone phase by developing some more core plugins, and port the legacy PlanetMath content to the new platform; (K2) add tools for authoring and solving interactive exercises, as well as tools for generating recommendations based on data about the learner's performance and prior knowledge; (K3) add various other useful plugins that make the site useful and attrac- p. 10

tive (e.g. integrating the SAGE computer algebra system, the Geogebra diagram-creating system, and a mathematics-enabled version of Etherpad); p. 11

(K4) to move in the direction of increased compatibility with other ways of interacting with mathematics online (e.g. compatibility with Wikipedia, ArXiv, Mizar). p. 11

Maria Druojkova recently reviewed two new brand new beta programs and ve others, looking at howtheystackupinseveralkeydimensions(seeFigure5).7 Itcertainlydoesn't seem impossible for a given website to have all of these features, and, indeed, PlanetMath seems to do fairly well at balancing the various desiderata, though there is always room for improvement. (In particular, tagging support has been in the works for far too long.) p. 12

One of my favourite dilemmas along the lines of Table 5 is the dierence between distributed innovation and centralized coordination. It seems possible that thanks to small world eects, PlanetMath's relatively early entry to the online mathematics space will help it remain somewhat central [38]. p. 12

7 http://groups.google.com/group/mathfuture/browse_frm/thread/9439bdcda8d67b52 p. 12

Will people want to do exercises online? Will people want to contribute new exercises? Will anyone apart from the existing group of developers be interested in working on the software? p. 13

How are people learning and doing mathematics? What are their frustrations? What can we do to improve the situation? p. 13

Even within the broader view expressed by this set of follow-up questions, the fact remains that the time (or other conditions) may not be right to crowdsource a PLE for mathematics. Although it is something of a last resort, a negative result like this could be interesting, if we can understand why we are seeing the negative result of this sort. We will turn to one such interesting negative result in the following section. p. 13

5 Pilot Study p. 13

In [37], we remarked: We do not view the Planetary System (or the KWARC stack for that matter) as an eLearning system in its own right, but rather as an enabling technology for eLearning in the STEM disciplines. An eLearning platform additionally requires pedagogy. p. 13

DIY Math through the Peer 2 Peer University (P2PU) p. 13

My course was meant to test these waters, and also to address a simplied variant of the research question from Section 3, along the lines of How should we build a course that supports both independent and peer-based mathematics learning? Although the course itself hasn't gone so well in my view, reections about the course have given a lot of useful ideas towards answering that question. p. 14

From the point of view of my broader research agenda, this course was a bit like designing a website on paper before writing code. Here, the design was for learning interactions. Specically, DIY Math was designed to build independent study and peer-support skills for mathematics learners at all levels. This is very dierent from a traditional mathematics course, which assumes that participants have approximately the same level of prior knowledge about p. 14

the subject, and furthermore that they will all follow a fairly similar learning trajectory. By contrast, the syllabus I created featured the open-ended exercises summarized here: p. 15

  • be ready to introduce yourself and your interests in the mailing list by the time the course starts;
  • reply to at least one other person's self-introduction to say how it relates to your own interests;
  • come up with some discussion guidelines or objectives;
  • update the group when you start looking at a new mathematical resource;
  • give a review saying whether you found that particular resource helpful or not, and why;
  • post short summaries of any study strategies you're using;
  • post reviews of your strategies;
  • identify one or more study buddies who are interested in topics similar to the ones you're most interested in. p. 15

Sounds very much like challenges - like Jessy's DIY researcher challenges or the DIYU… I still wonder how successful these kind of generic courses can be - it might be much more exciting to congregate around a certain topic or task, even though there might have to be many groups so that you can find one that fits your interest, level of knowledge, way of collaborating etc p. 15

Twenty two people initially signed up for the course, and their applications alone were enough to generate some useful reections. (See Appendix C for a summary of the mathematical interests the participants expressed in this phase.) p. 15

However, less than half of these people made it past the hurdle of self introductions, which were to take place in a Google Group I set up for the course.11 By the second week, things had gotten pretty quiet, and I gured we needed another exercise that would show people what they were meant to be doing on an ongoing basis. Accordingly, I added this to the syllabus, along with some description: p. 15

G post in response to one of the two prompts that follow: Tell us about something you learned.; and/or Tell us about something you're having diculty with. p. 15

11 http://groups.google.com/group/diymath p. 15

5.2 Post-Mortem Analysis/Pre-Mature Autopsy (PMA) p. 16

When I saw that things had gone dead quiet and my belated eorts to get people to engage with the core exercise G weren't going anywhere, I wrote up a detailed post-mortem analysis of the course (see Appendix D), which I shared with participants and also with the other organizers at P2PU. This spawned further discussions in both the diy-math and p2pu-community mailing lists, with one course observer declaring she's not dead, one participant saying he'd been in the hospital for a week, and another saying he'd been busy starting a corporation. Some more mathematical questions have been discussed on the list since then, the course was not the lively environment I had initially hoped it might be. Perhaps the discussion is only mostly dead, but it is certainly moving at a very slow pace at present. p. 16

In the mean time, the analysis plane took on a life of its own, and various ideas about how to design a peer-based course seem to constitute the most promising outcome of this experience (details coming up shortly). Now, it is certainly entirely possible that I could have taken the course to higher levels by being a more active participant myself. But like the other participants, despite the inherent interest of the subjects I had chosen to look at ([39] and [40]), it was easy for me to come up with other things to do. Since everyone in the course seemed to be in the same boat here, I don't blame my levels of participation, but some more fundamental design aws. p. 16

To briey sum up the take-home points from the experience as I see them: (1) when organizing an event or setting up a context for any given social activity, it is a really good idea to ask people what they would like to do in this context; and (2) it is also a really good idea to get the participants to commit to follow through on actually doing this stu to a reasonable degree. DIY Math was successful at (1), though I think this could have been done in a more ecient way. It seems to have utterly failed with regard to (2). p. 16

5.3 What to do dierently next time: part 1, SECI p. 16

Tables 14 present several dierent SECI-based analyses: (1) of the DIY Math course from the point of view of an idealized participant (i.e. how I thought things should work); p. 16

(2) of the DIY Math course from the point of view of a more realisticallyconceptualized participant (how it seems to me things actually worked); (3) of P2PU from the point of view of a facilitator (my own actual experiences); (4) a possible future analysis of P2PU from the point of view of an arbitrary participant. p. 17

The attening of roles in Table 4 is potentially surprising. I'm not insisting that the distinction between participants and facilitators go away, but a unied view can help us getting at the all-important shared context. p. 17

I say what I want to study We talk about diculties and successes Its discussions on a shared mailing list It helps me learn mathematics (and improve my skills at being a self-directed and peer-based learner) p. 17

Table 1: SECI analysis of DIY Math from the point of view of a somewhat idealized participant p. 17

I try to gure out what to study We sometimes give or get advice that isn't always so helpful (and most of it isn't for me, anyway) Its a bunch of good intentions that lead nowhere It conrms my sense of the diculty of learning anything in a self-directed fashion (and the diculty of mathematics in particular) p. 17

Table 2: SECI analysis of DIY Math from the point of view of a more realistically-conceptualized participant p. 17

If P2PU was taking an ongoing survey of the wished for course topics, I would presumably have been able to create a mathematics course tailored to the interests of pre-self-selected participants, instead of fashioning a course that (supposedly) served everyone with any interest in mathematics, simultaneously. I'm imagining a model sort of like Kickstarter12, wherein we would only run a course when enough interest had gathered. Levels of commitment could be set in advance, ranging from I'd like to put 1 hour a week into this course to I'd like to put 10 or more hours a week into this course. Detailed qualitative aspects of commitments could also be specied (e.g. I'm willing to answer  p. 17

I come up with a course I'd like to facilitate, and then facilitate it We discuss ideas about how our courses might work and what facilitation means (e.g. as opposed to teaching) Its discussions on a community mailing list and other settings (including discussions with participants in the course as it runs) It helps me improve my skills at a course designer and facilitator (and it's fun talking about and practising this stu!) Table 3: SECI analysis of P2PU from the point of view of a facilitator p. 18

I describe what I'd like to learn or teach We constitute a class when we have gathered a enough interested peers with the right mix of skills; in the mean time, and in general, we constitute a broader learning community Its a marketplace full of interests, skills, and ideas on the one hand, and eorts, attempts, and assistance on the other It helps me learn what I want to learn and have fun in the process Table 4: Possible future SECI analysis of P2PU from the point of view of any participant p. 18

Demand rather than supply model - could have different granularities, from "I just read this paper, would like someone else to read it and discuss it with me" to "I want to do this project for six weeks" p. 19

questions about Calculus or I know how to edit video). Pre-planning in this manner would help ensure that people knew what they were signing for, and also help everyone feel condent that they and their peers were making appropriate commitments. p. 19

The most obvious solution to the DIY Math problem is to not run such an unstructured course ever again! But we should be careful not to throw the baby out with the bathwater. Indeed, DIY Math as such might work quite well as a support system for learners who are also participating in more structured learning activities. It would be no problem to have four or more mathematics courses going at the same time, say, and have participants drop into a DIY Math backchannel for meta-discussions (cf. [41]). In this sense DIY Math would be more like a departmental seminar, or a seminar room and less like a course. The experience of a future DIY Math drop-in participant might indeed look like Table 1 but their course experience might be more like Table 5: p. 19

I work on calculus problems from the syllabus and ask questions in an appropriate venue when I get stuck We talk about diculties and successes and examples Its discussions on one or more shared mailing lists or forums It helps me learn calculus (and develop some good study habits along the way) Table 5: Possible future SECI analysis of Short Calculus from the point of view of a participant p. 19

Neat idea about Planet Math as a metastructure. I think this makes sense for the DIY research course etc as well… although not sure if course structure is right - more like StackOverflow? p. 19

Both this notion of a truly peer-led course and the earlier notion of an ongoing departmental seminar as a place to have meta-discussions get at ideas of a shared context in motion. Another related idea that came up in our post-mortem discussions was to generate a social contracts for courses with the assistance of a course mentor or godparent who might not be a subject-matter expert in the subject discussed in the course, but who had a specic expertise in running peer-to-peer courses. This might be someone who had previously helped run a successful course. This activity might take place in the rst week after a course quorum of interested participants had been found. Such would go along with re-thinking and nurturing the social contracts implicit to the P2PU structure p. 19

itself. I imagine discussions with and among these people to be at the highest meta-level, from the point of view of most course participants. Why, when, and how should people engage these dierent meta-levels? Another lasting outcome from reections on the DIY Math course may suggest some answers. This outcome was an outline of a pedagogic framework that applies to peer-to-peer or peer-based teaching-and-learning-between-equals. Since pedagogy isn't necessarily the right word to apply in this context, I created a new word, paragogy. p. 20

4. Learning is distributed and nonlinear. Side-tracking is OK, but dissipation isn't likely to work. Part of paragogy is learning how to work one's way around in a given social eld. p. 20

As a brief application: these principles suggest that people should engage with meta-levels in order to learn whatever it is that they want to learn about p. 20

This seems to make sense - we had problems with the balance between meta discourse and learning discourse in the CSCL course p. 21

learning, but not to such an extent that it dissipates the energies they might have put into learning about whatever they really wanted to learn about in the rst place! Confused? Don't worry: the next section is about sensemaking. p. 21

(1) How people can people claim to know what they're interested in doing, and still not follow through with it? p. 21

Perhaps one doesn't actually know what's involved, or perhaps there are other more pressing things going on. In a way, the latter is a sub-category of the former, insofar as one doesn't really understand the contextual features of one's own life. p. 21

It doesn't really make sense to blame participants for not knowing themselves well enough or not showing enough initiative as independent learners (this is what the DIY Math course was supposed to help people develop). Nor does it necessarily make sense to blame myself in my role as course facilitator for not having the ability to instill self-directedness in the other course participants. The temptation is then to blame the organizational structure of P2PU, e.g. for not running according to the model summed up in Table 4 but of course P2PU is itself undergoing organizational learning, and we didn't have a model like Table 4 until well after the DIY Math course was seen as done for! In short, DIY Math seems to have been a very useful learning experience, but mostly on the meta-level. This is important; as was remarked by another researcher, Marisa Ponti, who is also looking at P2PU: As far as I know, it is not something that we nd often in the literature, probably also because we often tend to examine `successful' rather than `not-successful' experiences.15 p. 21

(2) Why, when, and how should people engage these dierent meta-levels? This seems an appropriate place to remark that G (which you may recall from Page 15) perhaps somewhat inarticulately expresses a notion very similar to Dervin's Sense-Making Triangle of situation, gap, help. (From page 224 of p. 21

15 http://groups.google.com/group/p2punewcourses/browse_frm/thread/ 816b559efe02474a p. 21

the researcher is mandated to attend to what is called the Sense-Making Triangle: how the respondent sees the situation, what gaps the respondent sees [them]self as facing and/or bridging, and what ways the respondent saw [them]self as helped by the bridge he or she built.) In other words, the idea behind G was both to encourage the course participant in a process of making sense of their own experience, and also to bring in an explicit social dimension as part of this (now shared) sensemaking process (cf. [32], pp. 3843). The intuition here was based in part on my own experience in which many times, the act of formulating a question can itself lead directly to an answer (cf. [32], p. 15), and otherwise, a knowledgeable person is often of help. As I mentioned, the exercise in question wasn't even posted until the second week of the course: one thing to do dierently next time would be to share some notes on communication strategies with other participants right away. p. 22

But the point of mentioning this example here is that the very act of asking a question can take you to a meta-level. This is similar to the notion of the dierence between an automatic system (or doer) and a reective system (or planner) ([46], cf. [47]). Using one to the exclusion of the other would be a bit strange (the exemplars given in [46] are Homer Simpson and Mr. Spock). To come at this a dierent way, in any given situation we can stick with what works (regardless of how badly it works), or attempt to improve the situation (although we might not be sure exactly how to do that). The dilemma is nicely expressed here: p. 22

In general, it is not an exclusive choice between one extreme and the other (e.g. between organizational learning or personal learning), but, rather, the question is, what blend is right. Indeed, core and periphery bring to mind the idea of a Zipf/Pareto/power law. Not only do core and periphery fade into one another, they are in a certain sense almost reversible in this model [49]. Like other online communities (and many other aspects of the internet [50]), PlanetMath exhibits the familiar power law: a few people contributing most of the content, and many other contributors adding a little bit. p. 22

While we can easily get this out of PlanetMath's scoring mechanism, but almost all contributions at the meta-level are invisible within this mechanism. That said, it's not hard to see who has been engaged with meta-level stu around the site or to count them on the ngers of one hand. All of these people have been highly engaged with content at one time and in one way or another, but a better analysis would say what content (and what corresponding power law). E.g. PlanetMath's organizational wiki, AsteroidMeta, has become a potentially interesting corpus in its own right. The exact relationship of these meta-discussions to PlanetMath itself has never been entirely clear. Such discussions would presumably distract people p. 22

from learning mathematics, if they were mixed in indiscriminately with the rest of the PlanetMath corpus. At the same time, if PlanetMath is to become an an eLearning platform, we need a way to talk about and apply pedagogies (and, probably paragogies), and this can certainly be a very meta activity. More generally, any activity involved in consciously structuring the environment will be likely to involve meta-discussions. We would do well to understand them better. As we've seen, structure can be a very good thing; but we must beware of overly rigid structures that implicitly only ask questions like these: What of the things we can do would you like us to do? What of the things we now oer do you use? Do you like us? Which of the things we do do you like? Sense-making methodology reader by Brenda Dervin [30] p. 23

5.5 Summary of results of the pilot study p. 23

Open-ended unstructured conversations are not a suitable replacement for structured learning. The question as to how learners really want to structure their learning remains open, to be examined further, on a case-by-case basis if need be. I have developed the idea of paragogy partly to call attention to the fact that the various stakeholders in learning exist within a dynamically changing social eld. p. 23

I'll be able to do several phases of SECI analysis, Sensemaking, Coding. This will be a very useful supplement to the usage data I'll be able to get out of our site(s) once they go live. p. 24

In the spirit of modularity that applies throughout this project, dierent recommendation strategies can be tried as dierent plugins. Ideally I would like it if individual learners could tweak and design their own recommendation systems (e.g. always show hints, never show hints, show practical problems I should be able to solve given my current skillset, etc.). p. 24

17Early notes by Matthias Palmer and myself on the topic of widgets and spaces are at http://bit.ly/roleWidgetsAndSpaces. p. 25

7 Conclusion p. 26

I believe the plan outlined here has the potential to change the way mathematical culture works, along the lines suggested by Buckminster Fuller, by being a new model that makes the existing model obsolete. I'm thinking now of the teachers who say but if so much mathematical knowledge is online, won't students just use it to cheat? My response is: I hope they will use it to learn. But to reassure those who worry about cheating: this system should help people not only to know what they think, but also to know what they know. p. 26

I do wish to stress that this system doesn't make teachers, tutors, or experts obsolete: rather, I sincerely hope that it will help peer learners become teachers, tutors, and experts more easily! p. 26

It shows how to make a large-scale learning system sustainable. We briey discussed two dierent approaches to CBPP in education in Section 2.1: (1) taking a CBPP platform oine to use it in a course [7]; and, (2) asking members of a course to participate in a live platform [8]. p. 27

It is my futuristic hope is that my project will feed into subsequent investigations in the eld of articial intelligence. We would then ask questions like: How does the system actively shape or determine the roles of its users? What sorts of heuristics does it generate and deploy? p. 28

Adding to an already rich life, my father decided in middle age to become an elementary-school teacher in a working-class neighborhood in New Mexico. […] My father would have been unable to teach to the test. He once complained about errors in a sixthgrade math textbook, so he had the class learn math by designing a spaceship. My father would have been spat out by today's test-driven educational regime. […] To the degree that education is about the transfer of the known between generations, it can be digitized, analyzed, optimized and bottled or posted on Twitter. To the degree that education is about the self-invention of the human race, the gargantuan process of steering billions of brains into unforeseeable states and congurations in the future, it can continue only if each brain learns to invent itself. […] Roughly speaking, there are two ways to use computers in the classroom. You can have them measure and represent the students and the teachers, or you can have the class build a virtual spaceship. Right now the rst way is ubiquitous, but the virtual spaceships are being built only by tenacious oddballs in unusual circumstances. More spaceships, please. Does the Digital Classroom Enfeeble the Mind?, by Jaron Lanier, New York Times Magazine for September 16, 2010. p. 28

A A Brief History of PlanetMath The PlanetMath Story As told by Aaron Krowne19: p. 29

PlanetMath was conceived by Nathan Egge (unlord on PlanetMath) and Aaron Krowne (akrowne on PlanetMath yours truly) as a project to ll the void for a central repository for mathematical knowledge on the web, with a pedagogical slant. But we realized that we could do it in a way that would make it a community, built by the users, and that the users would be just as valuable as the content itself. There are now many grassroots-content sites like this (Wikipedia, Citeulike, Slashdot, Fark…), and PlanetMath represents a unique kind of hybrid of MathWorld, Wikipedia, and Slashdot concepts. We think that the end result has proven very useful to many people who are interested in learning math, who are already students studying math, or who make their living in mathematical professions. The idea for the project was hatched by Egge, sometime around fall 2000, in response to the unfortunate removal of the great resource, Eric Weisstein's World of Math (or MathWorld) from the internet. This incident created a void for a useful, comprehensive math encyclopedia freely available online, which we wanted to ll as quickly as possible. So Nathan and Aaron began brainstorming the concept and working out many of the problems inherent in a system that would do in real-time what had been static on MathWorld. We also wanted to make certain that the users of PlanetMath would never have to worry about the removal of the content they had contributed in favour of commercial sale. So it is the express goal of PlanetMath to be an internet resource, rst and foremost. We hope to achieve this with the aid of the GNU FDL, or Free Documentation License20. Via the FDL, we ensure that the contributors retain the rights to their work. Development began slowly after the initial ash of inspiration; as both Nathan and Aaron were very busy with school. Finally in the summer of 2001 we both graduated, and work really picked up pace. The core functionality of PlanetMath - the encyclopedia - is essentially complete. The site, however, continues to expand in scope (and it will probably never be completely done) and functionality as we continue to add new ideas to it in order to enrich and strengthen the PlanetMath community. p. 29

The Later-day PlanetMath Chronicles As told by me (jac on PlanetMath, formerly apmxi on PlanetMath; and jcorneli on AsteroidMeta).

B A SECI analysis of education p. 31

The table at left indicates the results of a tentative survey of the various activities that take place across the education sector. I've used simple glosses from Ken Wilber [52] instead of the harderto-remember Socialization, Externalisation, Combination and Internalisation phases of Nonaka and Toyoma. The idea is that education as a whole is comprised of activities like those in the table. The table could grow to become more detailed, but already we have moved from a vague picture of students, teachers, and administrators to a much richer sense of the dynamics involved across a variety of stakeholder groups. p. 31

gain a skill help a student generate knowledge create a tool/workow create a new identity have a fruitful exchange create a richer context be authoritative know the relevance p. 31

It p. 31

build a transcript give feedback write a paper create a system assemble strategies eect transaction(s) coordinate activities complete paperwork describe a skill set p. 31

Its p. 31

A given education platform (whether a university or a website) may support only some of the items in a table like this, in which case we can imagine those items lighting up. A combination of platforms will support generally more. For example, we can imagine a scenario in which PlanetMath provides content and supports peer-interactions, and P2PU provides a certication layer. A more detailed analysis of any given platform could take the table here as a model. At present, at least in the mathematics domain, it seems that free/open online support for the learning activities listed here is rather limited, or at least very scattered. A certain degree of cohesion in the learning platform will help to create a shared context in motion, which will make future analyses along these lines more useful. p. 31

Double-click to edit. p. 31

do a class project plan the curriculum discuss/experiment manage a body of work nd common ground trial phase/discussion negotiate relationships build credibility achieve a consensus p. 31

We p. 31

go to class lead a class ask a question share ideas/code engage in dialog describe/demo agitate/evangelize check work discern needs p. 31

I p. 31

D DIY PMA The following is a reformatted but otherwise unedited version of my Post-Mortem Analysis of DIY Math.21 The message was also forwarded to P2PU's main Community mailing list, where it spawned further discussion.22 p. 33

My assessment of the course was that participants knew more or less what they wanted to focus on, but they didn't exactly know how to start, and they didn't particularly want to discuss questions about how to do things. Unfortunately, how questions were supposed to be the central focus of this discussion-based DIY course. p. 33

At this point, I can see several profound shortcomings in the way I designed the course, not the least being that the implied tension between doing it yourself and discussing it with others was not given enough attention up front. I had hoped this would be a productive tension or dialectic, but it doesn't seem to have turned out that way. There is a certain degree of commitment required to work with(in) such a dialectic, and no one who signed up for the course seemed particularly interested in that. Rather, applicants for the most part just wanted to learn math. I did not do a very good job explaining why I think working with the diy/p2p dialectic is how to learn. p. 33

21 http://groups.google.com/group/diymath/browse_frm/thread/5fcb82598445cd54 22 http://groups.google.com/group/p2pucommunity/browse_frm/thread/ d0483169d626de10 p. 33

Aside from these issues, which I take responsibility for, some of the shortcomings of the course itself point to design considerations that would best apply at a higher level; something like P2PU best practices. One could say that courselevel and P2PU-level issues have some degree of trade-o between them. p. 34

guess is that either we'd need a more step-by-step curriculum crafted to ease people into thinking about how they learn, or else we'd need to know in advance that participants were really committed to asking and answering how questions. p. 34

24For example, I think it would be possible to design a Discovery Learning course where the very foundations of the course were to be discovered on the y by beginning students, but I think we should acknowledge that this would be going far beyond the standard Discovery Learning programme for mathematics education, which asks students to prove certain results from certain axioms, thus providing at least a basic logical scaolding; cf. [54] p. 34

25Alison recommends, for building community, and creating social contracts, Building Community in Schools by Thomas J. Sergiovanni [55], and, Two Bits by Christopher Kelty [56]. p. 35

3. eMath 3.0: Building Blocks for a Social and Semantic Web for Online Mathematics & eLearning (with Catalin David, Deyan Ginev and Michael Kohlhase), in Ion Mierlus-Mazilu, ed., 1st International Workshop on Mathematics and ICT: Education, Research and Applications, Bucharest, Romania (2010, to appear). p. 37

4. Semantic adaptivity and social networking in personal learning environments. Contribution to the WikiSym 2010 Doctoral Consortium. p. 38

1. Out from the shadow of Wikipedia: PlanetMath at Ten, talk given at Wikimania 2010. p. 38

An extended abstract has already been accepted for Crowdsourcing Education: A Collaborative Knowledge Management Approach, jointly written with Alexander Mikroyannidis, to appear in Dr. Alexandra Okada, Ms. Teresa Connolly and Dr. Peter Scott, eds., Collaborative Learning 2.0: Open Educational Resources. This book chapter will present a detailed analysis of PlanetMath using the SECI model (the discussion in Appendix B and the abstract of my talk Out from the shadow of Wikipedia: PlanetMath at Ten from Appendix F form the core of this chapter). p. 40

3. How to Hack It (in the real world) is a planned short book that takes Pólya's classic How to Solve It as a jumping o point, and then develops various heuristics that are suitable for work in a contemporary, social, online setting. Brief descriptions of the ten planned chapters are nished.28 I'd like the book to about the same size and format as Pólya's (a 250300 page mass-market paperback). According to my schedule of a chapter a month, a rst draft should be done in July or August of 2011. MathOverow.net's How to ask is an example of the sort of some of the strategy I'd like to discuss in the book.29 However, the book is not mathematicsspecic, rather, it is meant to expand generally on the idea of hackability described in Section 2.1, focusing on the relevance of this idea for learning and for commons-based peer production. 4. p. 40

References p. 42

[2] S. Wilson, O. Liber, M. Johnson, P. Beauvoir, P. Sharples, and C. Milligan: Personal Learning Environments: Challenging The Dominant Design Of Educational Systems. Proceedings of 2nd International Workshop on Learner-Oriented Knowledge Management and KM-Oriented Learning, In Conjunction With ECTEL 06, (2006) pp. 6776 p. 42

[3] Cormac Lawler: Coconguring Expansive Learning and Action Research in Wikiversity. Abstract submission: Mini-Conference on Medical Education, Learning Theories and Technology p. 42

[5] Aaron P. Krowne: An Architecture for Collaborative Math and Science Digital Libraries. Master's thesis, Virginia Tech (2003). http://scholar. lib.vt.edu/theses/available/etd-09022003-150851/ p. 42

[6] Joseph A. Corneli and Aaron Krowne: A scholia-based document model for commons-based peer production. In Martin Halbert, ed., Symposium on Free Culture and the Digital Library (2005), pp. 241254. p. 42

[7] Robert Milson and Aaron Krowne: Adapting CBPP platforms for instructional use. In Martin Halbert, editor, Symposium on Free Culture and the Digital Library (2005), pp. 255272. p. 42

[8] Piotr Konieczny: Teaching with Wikipedia and other Wikimedia Foundation Wikis, in Proceedings of the 6th International Symposium on Wikis and Open Collaboration (2010). p. 42

[11] Jacques Carette and William Farmer: A Review of Mathematical Knowledge Management, in Jacques Carette, Lucas Dixon, Claudio Coen and Stephen Watt, eds. Intelligent Computer Mathematics, LNCS 5625 (2009). p. 42

[12] Erica Melis: Errors as a Source of Learning in Mathematics, in Cognition and Exploratory Learning in the Digital Age (CELDA) (2004). p. 42

[13] Paul Libbrecht and Christian Gross: Authoring LeActiveMath Calculus Content, Mathematical Knowledge Management, MKM'06, LNAI (2006) p. 42

[17] C. Ullrich, Q. Chen, S. Chen, L. Shen, R. Shen: Investigating the reuse of course generation knowledge, In Proc. of the 15th Int. Conf. on Computers in Education (ICCE2007), Hiroshima, Japan, IOS Press (2007) p. 43

[18] E. Melis, E. Andrès, J. Büdenberder, A. Frishauf, G. Goguadse, P. Libbrecht, M. Pollet, and C. Ullrich: A generic and adaptive Web-based learning environment, International Journal of Articial Intelligence in Education 12 (2001) p. 43

[20] Barry J. Zimmerman, Albert Bandura, and Manuel Martinez-Pons: SelfMotivation for Academic Attainment: The Role of Self-Ecacy Beliefs and Personal Goal Setting, American Educational Research Journal, 29 (3), (1992) pp. 663-676. p. 43

[21] Dale Schunk and Barry Zimmerman, eds.: Motivation and Self-Regulated Learning: Theory, Research, and Applications, New York: Routledge (2007). p. 43

[22] Frank Pajares: Motivational Role of Self-Ecacy Beliefs in Self-Regulated Learning, in Dale H. Schunk and Barry J. Zimmerman, eds., Motivation and Self-Regulated Learning: Theory, Research, and Applications (2008). p. 43

[23] Albert Bandura: Exercise of personal and collective ecacy in changing societies, in Alfred Bandura, ed., Self-ecacy in changing societies (1995) p. 43

[24] Ivan Illich: Disabling Professions: Notes for a lecture, Crime, Law and Social Change 1(4), (1977). p. 43

[26] Lars Svensden: The Philosophy of Boredom, Reaktion Books Ltd. (2005). (Translation of Kjedsomhetens loso, 1999.) p. 43

[27] Masao Abe: Nishida's Philosophy of `Place', International Philosophical Quarterly 28 (4), (1988) pp. 355371 p. 43

[28] I. Nonaka, R. Toyama, N. Konno: SECI, Ba and Leadership: a Unied Model of Dynamic Knowledge Creation Long Range Planning, 33(1), (2000) pp. 534. p. 44

[29] Bonnie A. Nardi (ed.): Context and Consciousness: Activity Theory and Human-Computer Interaction, The MIT Press (1996) p. 44

[30] Brenda Dervin: Sense-making methodology reader: Selected writings of Brenda Dervin. Cresskill, NJ: Hampton Press, Inc. (2003) p. 44

[31] Gilles Deleuze: The logic of sense. Continuum Impacts (2004). (Translation of Logique du sens, 1969.) p. 44

[32] Karl E. Weick: Sensemaking in Organizations. SAGE Publications (1995). p. 44

[33] G. Klein, B. Moon, and R.F. Homan: Making sense of sensemaking I: alternative perspectives. IEEE Intelligent Systems, 21(4), (2006). pp. 7073. [34] G. Klein, B. Moon, and R.F. Homan: Making sense of sensemaking II: a macrocognitive model. IEEE Intelligent Systems, 21(5), (2006). pp. 8892. p. 44

[35] George Pólya: How to Solve It, Princeton University Press (1945) p. 44

[36] E. Melis and C. Ullrich: How to teach it: Pólya-scenarios in ActiveMath, In U. Hoppe, F. Verdejo, and J. Kay, eds., Articial Intelligence in Education, IOS Press, (2003) pp. 141147. p. 44

[37] Catalin David, Deyan Ginev, Michael Kohlhase and Joseph Corneli: eMath 3.0: Building Blocks for a Social and Semantic Web for Online Mathematics & eLearning, In Ion Mierlus-Mazilu, ed., 1st International Workshop on Mathematics and ICT: Education, Research and Applications, Bucharest, Romania (2010, to appear) p. 44

[38] Filippo Menczer: Growing and navigating the small world Web by local content, PNAS 99(22), (2002) pp. 1401414019 p. 44

[40] Vladimir N. Vapnik: Statistical Learning Theory Wiley-Interscience (1998) p. 44

[42] Kapp, Alexander: Platon's Erziehungslehre, als Pädagogik für die Einzelnen und als Staatspädagogik. Minden und Leipzig. (1833) p. 44

[51] Mohan Ganesalingam: A Language for Mathematics. Ph. D. thesis, University of Cambridge (2009). http://people.pwf.cam.ac.uk/mg262/ p. 45

[52] Ken Wilber: An Integral Theory of Consciousness, Journal of Consciousness Studies, 4(1), (1997) pp. 7192 p. 45

[53] M. Myllyaho, O. Salo, J. Kääriäinen and J. Koskela: A review of small and large post-mortem analysis methods, In Proceedings of the ICSSEA, Paris, 2004. (http://www.vtt.fi/merlin/pub/pma_full_1.00-icssea-layout. pdf) p. 45

[54] W.S. Mahavier: What Is The Moore Method?, Primus, vol. 9 (December 1999), pp. 339254 (http://legacyrlmoore.org/reference/mahavier1. html) p. 45

[55] Thomas J. Sergiovanni: Building Community in Schools, San Francisco: Jossey-Bass (1994) p. 45

[56] Christopher M. Kelty: Two Bits: The Cultural Signicance of Free Software, Duke University Press (2008) (http://twobits.net/read/) p. 45

[57] I. Nonaka and R. Toyama: The knowledge-creating theory revisited: knowledge creation as a synthesizing process, Knowledge Management Research & Practice, 1(1), (2003), pp. 210. p. 45

[58] Saverio Perugini, Marcos André Gonçalves, and Edward A. Fox: Recommender Systems Research: A Connection-Centric Survey, Journal of Intelligent Information Systems, 23(2), (2004), pp. 107143. p. 46