Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings

Throughout my career, I have been interested in the problem of learning to learn, a topic with an honorable history in psychology and education. Educationalists have been interested in the problem because of two stumbling blocks to lasting learning: (a) inert knowledge (Whitehead, 1916): students acquire facts that they cannot access and use appropriately; and (b) passive learning: students do not readily engage in intentional, self-directed action. These two problems are commonly regarded as diseases of schooling (Brown, 1977) p. 4

Like many experimental and developmental psychologists operating during the 1970s, my research focused on human memory: We avoided the term p. 4

learning because of its association with behaviorist theories. During this period, there was an important shift in how we characterized children's purportedly poorer memories; we progressed from blaming poor perfor­ mance on limited memory capacity per se to concentrating on the role of active memory strategies (Brown, 1 974, 1975; Chi, 1 976, 1 978). This controversy led to the introduction of the training study as a theoretical tool to tease apart the relative contributions of capacity and strategic activity. The argument took the form of a question: Do young children fail to use strategies because they do not think to do so, or do they suffer from a mediational deficiency such that strategic intervention would not help them anyway? The response to this theoretical question was to train children in the use of strategies to see if their performance would be mediated effectively once they applied appropriate strategies. In information pro­ cessing terms the distinction was between voluntary control processes (strategies) and structural features (memory capacity), a distinction that also led to attempts to distinguish the two through training (Atkinson & Shiffrin, 1968; Brown, 1975). Note, however, that this was an issue of theory, an attempt to unravel mysteries of the memory system. The work had nothing to do with theories about how to make a better learner. The players had absolutely no educational relevance in mind p. 5

The main findings of 10 years' work can be summarized in two statements: 1 . Training worked, even given the pathetic amounts provided in the typical study. Children could be trained to use simple strategies, and when they did use them, memory improved. 2. Training didn't work, because there was little evidence of maintenance in the absence of the experimenter's prompting and less evidence of independent transfer (Brown, 1978; Brown & Campione, 1 978). p. 5

A major impetus to my early work was a review in which a leading theoretician characterized the vast body of memory research as trivial at best, and pointed out that psychologists do not study the most interesting thing about human memory, the fact that people have knowledge and beliefs about it (Tulving & Madigan, 1970). This topic became known as metamemory (Flavell, 197 1 ), and it was enthusiastically studied by devel­ opmental psychologists because the problem of inert knowledge and passive learning appeared to be more prevalent in the young. p. 6

My research agenda in the 1970s was governed by these questions: Why is it that young children do not use strategies? Is it that they do not know that such activities are useful, or that they do not care, or a little of both? Enter metacognition: knowledge about and control of one's own learning. Studies of metacognition dominated the latter half of the 1970s, even though it was agreed that the term was fuzzy and ill defined (Brown, 1975, 1978; Flavell & Wellman, 1977; Wellman, 1983). Again, to summarize the outcomes crudely: Children do not use a whole variety of learning strategies because they do not know much about: (a) remembering (they know little about the strategies and tactics of overcoming memory limitations) and (b) monitoring (they do not think to orchestrate, oversee, plan, and revise their own learning activities). The terms remembering, monitoring, strategies, and metacognition widely replaced memory, denoting an important theo­ retical shift from passive to active metaphors of learning. This shift was attended by a need to develop new methods to capture active strategic and monitoring processes (Brown & Campione, 1979; Schoenfeld, 1 985). p. 6

During the 1970s, psychologists interested in learning and memory began shifting from the almost exclusive study of the learning of lists of words, pictures, and paired associates to a concentration on coherent content. They began also to look at the acquisition of expertise within a domain- gained over long periods of time via concentrated, and often self-motivated, learning (chess, for example). More recently, learning theorists have considered the acquisition of disciplined bodies of knowledge, characteristic of academic subject areas (mathematics, programming, physics, and even some areas of social science). Learning theories had to change to reflect this (Glaser & Bassok, 1989). My own research also reflected a shift to the study of learning, remembering, and understanding complex texts, which in turn led to studies of reading comprehension and comprehension monitoring in specific content areas. p. 7

Then, using the Jigsaw method, the students regroup into learning groups in which each student is expert in one subtopic, holding Ys of the information. Each one-fifth needs to be combined with the remaining fifths to make a whole unit, hence “jigsaw.” The expert on each subtopic is responsible for guiding reciprocal teaching seminars in his or her area. Thus, the choice of a discussion leader is now based on expertise rather than random selection, as was the case in the original reciprocal teaching work. All children in a learning group are expert on one part of the material, teach it to others, and prepare questions for the test that all will take on the complete unit. During this cyclical process, students both acquire content expertise· and learn how to learn from texts and other media (Brown & Campione, 1990). p. 11

One of the major decisions psychologists must make is to decide on the “grain size” of the phenomena they wish to observe. The classic distinctions are laid out neatly in introductory textbooks. Developmental psychologists are informed of the strengths and weaknesses of several such options . First there is the choice between an idiographic and a nomothetic approach, that is, few or many. Does one want to study a “single variable in many subjects for the purpose of discovering general laws or principles of behavior” or p. 14

adopt the idiographic approach, “the thorough study of individual cases, with emphasis on each subject's characteristic traits. ” p. 15

The Hawthorne Effect, as it appears in standard texts, refers to the fact that any intervention tends to have positive effects merely because of the attention of the experimental team to the subjects' welfare. p. 23

Discovery learning. A second popular tenet of Dewey's (1 929) peda­ gogical creed is the concept of discovery learning. The children learn best when discovering for themselves the verities of life. This notion has been fully incorporated into constructivist theories of learning, and one can surely have little critical to say about it. p. 28

Children “discovering” in our biology classrooms are quite adept at inventing scientific misconceptions. For example, they readily become Lamarckians, believing that acquired charac­ teristics of individuals are passed on and that all things exist for a purpose (teleological reasoning). They overdetermine cause, thus blinding them­ selves to essential notions of randomness, spontaneity, and chance. p. 29

We believe that the middle ground of guided discovery is the most appropriate for our classrooms, but this role is difficult to maintain. p. 29

The spirit of Dewey's position was intended to counteract the isolation of much of school learning from the familiar habits of childhood on the one hand, and adult occupations on the other (Cole & Bruner, 197 1); however, there is another side to the story. Schools came into existence to expose children to knowledge outside the realm of lived experience, an exposure rarely possible in preliterate society. As Bruner (1 969) argued, to under­ stand something is to relinquish prior ways of thinking, and he is skeptical about the practice of situating learning exclusively in the child's lived experience and social activities. For example, he argued that: The significance of the concept of commutativity in mathematics does not derive from the social insight that two houses with fourteen people in each is not the same as fourteen houses with two people in each. Rather it inheres in the power of the idea to create a way of thinking about number that is lithe and beautiful and immensely generative. . . . Without the idea of commutativity, algebra would be impossible. If set theory . . . had to be justified in terms of its relation to immediate experience and social life, it would not be worth teaching. Yet set theory lays a foundation for the understanding of order and number that could never be achieved with the social arithmetic of interest rates and bales of hay at so much per bale. (Bruner, 1 969, p. 1 2 1 ) p. 30

The excitement of teachers and students is palpable when it all works, when they experience what one of my teachers calls “golden moments. ” However, in reporting these moments, it i s important to realize that they are selected from a stream of events that are not so encouraging. One must separate the gold from the dross. But how does one represent the gold-dross ratio? Hence “lies” in the title; there is a tendency to romanticize research of this nature and rest claims of success on a few engaging anecdotes or par­ ticularly exciting transcripts. One of the major methodological problems is to establish means of conveying not only the selective and not necessarily representative, but also the more important general, reliable, and repeatable p. 33

REFERENCES p. 35