Knowledge Forum, the original Knowledge Building Environment [4], stores discourse, knowledge objects and artifacts, time-stamped information about activities and interactions, and revision histories in an extensible database [5]. Users of Knowledge Forum have had access to powerful analytic tools for nearly a decade. Early analytic tools provided access to tabular representations of data. We experimented briefly with tools to graph results, but in advance of embedding tools in a comprehensive framework for assessment, and creating visualizations that allow results to be integrated into the ongoing work of the community, their use was limited [6]. The current assessment framework supports concurrent, embedded, and transformative assessment, thus represents an important step forward. However, it remains difficult for the end user to synthesize the vast amount of information provided by these assessment tools. (loc: 11-18)

The aspects of knowledge advance that we are interested in measuring have historically been difficult to measure without engaging in time-consuming manual analyses: depth of understanding and explanation, idea diversity, curricular coverage. Additionally, the goal of transformative assessment requires that we present those results to users to inform their ongoing process, not simply to researchers or instructors to analyze results. Our goal is to employ complex yet comprehensible visualizations that convey the results of such assessments to participants as young as 5 and as old as 105. (loc: 18-22)

Our assessment questions include: in what contexts has a particular individual worked? What are the dominant ideas in the discourse space? What does the evolution of the discourse space over time look like? How similar are the writings of participants to those of experts? Can we identify depth of understanding and conceptual change? (loc: 23-26)

2. Latent Semantic Analysis A promising approach to finding answers to these questions lies in the use of Latent Semantic Analysis (LSA). LSA is a statistical technique used to extract the deep meaning of patterns of words in specific contexts of use. The technique is performed by applying methods from linear algebra (matrix decomposition and dimension reduction) to matrices that represent usage patterns of terms in [7,8]. It is also a theory about knowledge acquisition in human beings [9,10]. Current applications of LSA include indexing and information retrieval [11], assessment of text coherence [12,13,14], automated grading of essays [15], and summary scoring and revision [16]. (loc: 26-31)

References [1] M. Scardamalia, Technology for understanding, pp. 83–88 in: K. Leithwood, P. McAdie, N. Bascia, and A. Rodrigue (Eds.) Teaching for deep understanding: towards the Ontario curriculum that we need, Toronto, 2004. [2] C. Bereiter, Understanding technology, pp 48–5188 in: K. Leithwood, P. McAdie, N. Bascia, and A. Rodrigue (Eds.) Teaching for deep understanding: towards the Ontario curriculum that we need, Toronto, 2004. [3] Y. Sun, J. Zhang, and M. Scardamalia (in press). Developing deep understanding and literacy while addressing a gender-based literacy gap. Canadian Journal of Learning and Technology. [4] M. Scardamalia, and C. Bereiter, Knowledge building environments: Extending the limits of the possible in education and knowledge work. In A. DiStefano, K.E. Rudestam, and R. Silverman (Eds.), Encyclopedia of distributed learning, Thousand Oaks, CA: Sage Publications, 2003. [5] ZooLib. Available from http://zoolib.sourceforge.net/. Accessed 11 May, 2007. [6] C. Teplovs, Z. Donoahue, M. Scardamalia, and D. Philip, Tools for Concurrent, Embedded, and Transformative Assessment of Knowledge Building Processes and Progress, CSCL2007. [7] S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, and R. Harshman, (1990). Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 41: 391-407. [8] T. K. Landauer, P.W. Foltz, and D. Laham, (1998). An introduction to latent semantic analysis. Discourse Processes 25: 259-284. [9] T. K. Landauer and S. T. Dumais (1997). A solution to Plato's problem: The Latent Semantic Analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211-240. [10] W. Kintsch (1998). Comprehension: a paradigm for cognition. New York: Cambridge University Press. [11] S. Dumais. LSA and Information Retrieval. pp. 293-322 in: T. K. Landauer, D. S. McNamara, S. Dennis and W. Kintsch (Eds.) Handbook of Latent Semantic Analysis. Mahwah, NJ. (2007). [12] P.W. Foltz (1996). Latent Semantic Analysis for text-based research. Behavior Research Methods, Instruments and Computers 28(2): 197-202. [13] P.W. Foltz, W. Kintsch, and T.K. Landauer, (1998). Analysis of text coherence using Latent Semantic Analysis. Discourse Processes 25. [14] P.W. Foltz. Discourse Coherence and LSA. pp. 167-184 in: T. K. Landauer, D. S. McNamara, S. Dennis and W. Kintsch (Eds.) Handbook of Latent Semantic Analysis. Mahwah, NJ. 2007. [15] Landauer, T.K., Laham, D., & Foltz, P.W. (2003). Automatic essay assessment. Assessment in Education: Principles, Policy & Practice, 10(3), 295-308. [16] E. Kintsch, D. Caccamise, M. Franzke, N. Johnson, and S. Dooley. Summary Street: Computer-Guided Summary Writing. pp. 263-278 in: T. K. Landauer, D. S. McNamara, S. Dennis and W. Kintsch (Eds.) Handbook of Latent Semantic Analysis. Mahwah, NJ. (2007). [17] B. Shneiderman. The eyes have it: a task by data type taxonomy for information visualization. Proceedings of IEEE Workshop on Visual Languages '96 (1996), 336–343. (loc: 56-81)