Abstract. Some might argue that the analytics tools at our disposal are currently mainly used for boring purposes, such as improving processes and making money. In this paper we will try to define learning analytics and their purpose for learning and education. We will ponder on the best possible fit of particular types of research methods and their analysis. Methodological concerns related to the analysis of Big Data collected on online networks as well as ethical and privacy concerns will also be highlighted and a case study of the use of learning analytics in a Massive Open Online Course explored. p. 2

Lazer et al. [4] stress that social scientists have lagged behind researchers in other fields, for instance in fields such as biology and physics, and that it is unavoidable for analytics to become part of social science research. p. 2

Moreover, they emphasise the urgency for a data-driven computational social science to develop „based in an open academic environment‟, rather than in the domain of private companies such as Google and Yahoo, and government agencies who are currently the main players in the analytics field. They answer the question: “What value might a computational social science – based in an open academic environment – offer society, by enhancing understanding of individuals and collectives?” [4, p.721]. p. 2

Norris et al. [7] have a slightly different emphasis on the use of analytics; they would like analytics to be used to measure, compare and improve the performance of individuals, not just to better the experience but also to facilitate better outcomes to the activity. p. 3

Most analytics are related to the introduction of Learning Management Systems (LMSs) and are sometimes called Academic Analytics[8][9][10]. p. 3

Dawson et al. [9] added that the analysis of this data might be used to improve the student learning experience, which would not only require a quantitative analysis, but also a qualitative one, or at least a qualitative interpretation of findings. The interpretation would have to include a value judgment on people‟s use of the environment: not only counting who uses the environment for what, but also judging what might be a good and what might be a bad experience, and offering suggestions for moving on the continuum from one to the other. p. 3

There is a whole set of approaches having to do with content analysis. The idea is to look at contributions in discussion forums, and to analyze the kind of contribution. Was it descriptive? Was it on-topic? Was it evaluative? Did it pose a question?” [10] p. 3

Parry [11] and Kop [12] highlight possibilities to take this one step further, and suggest that analytics could be used not only to provide managers of learning and possibly educators and learners with information that they can use to improve learning, but also to provide learners with recommendations in their learning based on earlier learning activity. p. 3

De Laat [13] highlighted the complexity of researching networked learning and emphasized as key problems the issues of human agency and the multitude of issues involved, such as the dynamics of the network, power-relations on the network, and the amount of content generated. Effective analysis would require a multi-method approach. He suggested the use of computer-generated content analysis to explore what people are discussing. In addition, interviews with an emphasis on critical event recall focusing on the experiences of participants to find out „why they are talking as they do‟ and Social Network Analysis to find out the dynamics of the network to see „who communicates with whom‟ [13, p. 110]. p. 4

PLENK2010 p. 5

As vast amounts of discursive data were generated and collected, analysis and computational tools have been used to represent large networks of activity in the PLENK, to identify themes in the data and for analysis and interpretation of the qualitative research data (e.g., SNAPP, Pajek, NetDraw and Nvivo). p. 6

Social Networks Adapting Pedagogical Practice (SNAPP) uses information on who posted and replied to whom, and what major discussions were about, and how expansive they were, to analyze the interactions of a forum and display it in a Social Network Diagram. p. 8

Between 40 and 60 were active producers, the other 1580 were not visibly active. This was unexpected to the course organizers as before the start they saw the production phase as vital to the learning on a networked environment. After all, as some participants mentioned in the discussion, if nobody is an active p. 8

producer, it limits the resources that all participants can use to develop their ideas, to discuss, think, and be inspired by in their learning. p. 9

From a research point of view, the time and efforts needed to conduct various analyses on two forum discussions was prohibitive, but yielded a detailed view of what actually occurred in one discussion, including the processes, the learning, and important outcomes. The use of tags in the Moodle environment would have been helpful in linking various contents across weeks, allowing participants to search for relevant content and to see how they were connected to various content and people with similar interests. p. 9

Intelligent and automatic data analysis with powerful computational tools for analysis and interpretation should be explored as a valid option for informing learning in MOOC in a connectivist-type course. p. 10

References p. 11