The classroom flip (or inverted classroom) is one such innovative classroom structure that moves the lecture outside the classroom via technology and moves homework and practice with concepts inside the classroom via learning activities. p. 3

This research compares the classroom flip and the traditional lecture/homework structure in two different college level introductory statistics classrooms. In the classroom flip classroom, an intelligent tutoring system (ITS) was used to deliver the lecture content outside the classroom. Students completed active learning projects in the classroom that often required the use of a spreadsheet computer program to help students work with the concepts in the course. In the lecture/homework classroom, students attended lectures on course content that included PowerPoint slides, and then students practiced with the course concepts by completing homework from their books outside of class. p. 3

The learning environment and the learning activity in both classrooms are investigated in this study with respect to activity theory and learning environments research. Students were given the College and University Classroom Environment Inventory (CUCEI) to measure both their learning environment preferences and their learning environment experiences. p. 3

data were collected via field notes, p. 3

classroom transcripts, student interviews, student focus groups, researcher journal entries, and student reflections. The quantitative data were analyzed using t-tests and MANOVA, and the qualitative data were analyzed using grounded theory methods. p. 4

The findings of this research show that classroom flip students were less satisfied with how the structure of the classroom oriented them to the learning tasks in the course. The variety of learning activities in the flipped classroom contributed to an unsettledness among students that traditional classroom students did not experience. Finally, the concept of student comfortability with learning activity is presented and developed in light of learning environments research. p. 4

Many college and university professors desire to change their instructional style from traditional lecture to a more active, student-centered style through the use of group projects, discovery activities, experiments, and class presentations (Baker, 2000) p. 14

Description of how flipped classroom came to be: professors traditionally were interested in making lectures more interactive and collaborative but feared that they were sacrificing course content. With availability of online between-class systems, new possibilities p. 14

This change in how course content is introduced to and engaged by students is a significant departure from the lecture-homework cycle found in more traditional classrooms. Perhaps the most striking departure is the physical location of where the introduction and deeper engagement with the material occurs. Traditionally, the introduction is given in class through a lecture, and the deeper engagement occurs outside p. 14

of class through homework. In the above description, however, the introduction occurs outside of class and the engagement occurs inside the classroom. Researchers have come to dub this flipping of what is traditionally done inside and outside the classroom the “classroom flip” (Baker, 2000) or the “inverted classroom” (Lage & Platt, 2000; Lage, Platt, & Treglia, 2000). p. 15

As an instructor, had already begun experimenting with flipped classroom himself, although he didn't know what it was called/theory behind p. 16

Script for pilot: watch videos and answer quiz at home (if wrong on quiz, explanation displayed). In class, first quick quiz, then more challenging problems in pairs, roaming teacher. Only 15 students. p. 17

At the time I was teaching this precalculus course, I was also exploring research that dealt with technology and education. Themes of interest to me in this literature dealt with control over learning, student confidence with the computer, student enthusiasm, connections to people rather than machines, and the need for a physical classroom. I developed a questionnaire that addressed some of these themes (see Appendix B), and at midterm I administered this questionnaire to get feedback from students about their experience in this unique classroom setting. All of the questions on the questionnaire were categorical in nature, so I used the chi-square test statistic as a tool to look for significant relationships among these themes. p. 18

On the written portion of the questionnaire, half of the students stated they loved having control over the video lectures and freedom to choose when they would view lectures and complete homework. However, many students recounted problems with technology that hurt them in the learning process. p. 19

Many students mentioned how frustrating it was to not be able to ask the professor on the video a simple question, and almost all students at one point or another in the semester stressed how inconvenient it was to have to be connected to the Internet to complete their multiple choice homework assignments. p. 19

When designing a questionnaire about flipped teaching, also include technical difficulties, how hard it was to access the material, software etc. p. 19

Twelve students reported being enthusiastic about technology use at the beginning of the term while only 4 reported being enthusiastic about technology use at midterm. p. 19

In the questions that measured relationship development in the class, none of the students responded that they felt more connected in their relationships with the professor or other students in this class than they do in their other classes. In an even split, 7 students said they felt the same level of connectedness to other students in this class as compared to their other classes and 8 students said they felt less connected. However, students reported 11 to 4 that they felt less connected with the professor in this class than in their other classes. p. 20

Finally, the issue of the need for a physical classroom was raised in 3 of the students’ written responses and in conversations with students throughout the semester. Students commented that class was redundant, boring, or even a waste of time. As one student wrote, “Class can get a little boring since we already learn the stuff on the computer.” p. 20

Two conclusions from this pilot study informed my dissertation research. First, if a professor is going to use technology as the main tool to introduce students to new content, then the technology must work smoothly and not burden students with its form. p. 20

Second, it appears that using technology to introduce students to new content may cause students to feel less connected to the professor in the classroom. Using technology for the introduction may also negatively influence students’ ability to transfer their learning to contexts different from those in the initial introduction. When students feel class is a waste of time, this is an indication that the professor must offer something beyond more difficult practice problems in class. Perhaps the professor should focus classroom activity on helping students transfer their learning to new situations. Whatever the case, professors must offer something in the classroom that students cannot get elsewhere. They must create an environment where the classroom becomes a dynamic learning community. p. 21

During three days in February of 2002, I again investigated the classroom flip, this time in two of my Introduction to Statistics classrooms. I structured one class as a lecture and homework out of a book class and the other as a classroom flip class. I wanted to investigate how the change in communication in a classroom flip setting would influence students’ learning, confidence, and attitudes toward learning. p. 21

I used a random number generator to choose the 8:00 a.m. section of Introduction to Statistics to be the classroom flip class and the 9:10 a.m. section to be the lecture and p. 21

homework class. Both sections had been structured as a lecture and homework class when we learned the information in chapter 1 of the textbook. For chapter 2 (the descriptive statistics chapter), however, I had the flip class go to the library to check out and watch a 55-minute video of me lecturing over the information from the chapter for their homework. When students came to class, they completed a group project for three class periods. This project gave them an opportunity to apply descriptive statistical techniques to a real-world context. Students then wrote a report to a fictional character who asked them to put their mathematical findings in everyday language. (For a description of the project, see Appendix C). The lecture and homework class continued in the normal way for the three days that we discussed chapter 2. When this three-day investigation was complete, both sections returned to the lecture and homework class format for the remainder of the semester. p. 22

I recorded grades on the first exam (chapters 1–3), developed and distributed a survey to measure students’ confidence in their abilities to complete problems over the information in chapter 2 (see Appendix D), videotaped class sessions, and interviewed 3 students from the flip classroom. I analyzed the quantitative data using a t-test to look for significant differences between the two sections. I analyzed the qualitative data by performing a theme analysis to look for patterns that could provide descriptions and explanations of what was happening in the two classrooms. p. 22

An analysis of the confidence surveys showed that students from the lecture and homework class were more confident in their abilities to successfully complete a quiz p. 22

over information from chapter 2 than students from the flip class (p < 0.001). Students in the lecture and homework class also had a higher average score on the first exam than students in the flip class (p < 0.05). p. 23

Analysis of the classroom flip interviews revealed that students seemed to struggle with where they fit into this new way of doing class. As one student described, p. 23

The thing I have with [the classroom flip] really is that it didn’t fit my study habits. I’m used to doing my homework later at night with either a movie on or music playing. And with all of those people talking, especially some certain groups in the class that – it was just really distracting and it was just a big change and really out of my comfort zone. … I think with it being at 8:00 in the morning I’d rather hear a lecture than wake up and try and do my homework, because I’m not a morning person at all. And I can sit and write things on PowerPoint notes but especially math, because it’s like my weakest subject, I think I’d rather get a lecture in the morning. p. 23

Students were forced to adjust personal learning strategies they had relied on for years to fit this new classroom structure, and it appeared this adjustment was something students had difficulty doing in a short period of time. p. 23

Another theme found in the interviews dealt with the in-class working environment for students. One student mentioned not being a “work-in-groups kind of person” and said, “Our group pretty much just faced each other and then did our own p. 23

work. So that’s what we did, so that was fine for me.” I went to the classroom videotapes to see how prevalent this approach was and found that about two-thirds of the students sat silently in their groups working on their papers while one-third of the students discussed their courses of action with others. This lack of robust group discussion perhaps explains why so many of the students’ reports read more like a play-by-play account of the steps involved in finding the mean, median, quartiles, stem-and-leaf display, etc. rather than a creative report explaining what these statistics tell the reporter (the fictional character to whom they addressed their report) about the data under investigation. p. 24

This mental separation from the group is possibly the way many students best coped with the radical change they were experiencing in their working environment. They withdrew and then did their individual best even when they technically were required to work collaboratively. p. 24

Since the classroom flip structure was not going to extend beyond three days, students may not have seen the benefit of exerting the time and energy it takes to make a good group discussion work. p. 24

The data revealed an interesting link between learning in groups and confidence. When some students worked in groups and heard other people in the class explaining things in a different way, they began to feel like their thinking was wrong and doubted their abilities. Instead of engaging other students in conversation, they preferred to work alone and feel more confident in their work (even if the final product was not necessarily as good). p. 25

Three findings from this pilot study informed my dissertation. Students must have time to adjust to the changes the classroom flip brings, there must be flexibility so that students can become comfortable with the things they do and do not have control over in the learning process, and students must come to see that group learning activities can benefit their personal learning. p. 25

In most instances where the classroom flip is used, the professors explicitly state that they chose to use this format in order to give students a chance to actively engage course material without losing the coverage of course content (Baker, 2000; Bowers, 2002; Collins, Glenn, Violanti, & McKinney, 2002; Lage et al., 2000; OCTET, 2003; Phillips, 2002; Roane State Community College, 2003; Schauf, 2002; Thinkwell, 2002; Turoff, 1999; Ursuline College, 2002). p. 25

While the classroom flip may be 12 p. 25

useful in helping professors address this practical need in their classrooms, it may present theoretical problems for classroom learning. p. 26

The idea that course content can be “delivered” is founded on behaviorist theories of learning where knowledge is viewed as an objective entity that can be transferred from one person to another (as learned skills or strategies of thinking). However, active learning techniques have been used over the past few decades by educators who espouse constructivist theories of learning which view knowledge as something that must be built up by the learner through reflective abstraction. Thus, a classroom flip environment could end up being a place where the outside class activity is driven by one learning theory, and inside class pursuits are driven by a different (competing/conflicting) theory. It seems this situation could pose a significant problem for professors and students alike. As a result, I was interested in studying how the complexities inherent in the classroom flip method of structuring a course influences the learning environment for students as they progress through the course. p. 26

SELF: Various ways of introducing the research. One is to try to show that flipped classrooms are better etc, another is to say "people are doing flipped classrooms, however there are lots of problems, how can we make flipped classrooms better" - very different way of framing it p. 26

Learning environments can be conceived of as being made up of three domains: relationship, personal growth, and system maintenance and change (Moos, 1979, 2003). p. 26

Relationship dimensions involve student attentiveness in the classroom, the extent to which students are involved with each other and come to know each other in the class, how the students work with others in the class, and the professor’s interest and attitude toward the students. Personal growth dimensions involve how students work to complete academic tasks and how they are motivated to learn. System maintenance and change deals with how orderly the class structure is, how clear expectations are, how the environment is controlled, and how responsive the classroom is to change. p. 26

This study sought to investigate how the structure of the classroom flip affects the learning environment for students. In one classroom, I used a user-friendly intelligent tutoring system (ITS) called ALEKS to introduce students to course content outside the classroom, while we completed projects together inside the classroom. Students in this flip environment and students in a lecture-homework environment took surveys, were interviewed, and were observed in class. I analyzed the data from these two learning environments to better understand the implications of this practical push for technology use in college and university classrooms. p. 27

This chapter develops a conceptual framework (arranged according to Figure 2.1) for investigating learning activity in a classroom that is structured using the classroom flip. Extensive use of educational technology to deliver course content outside of class is central to the classroom flip idea (the arrow going to the left and downward in Figure 2.1). Active learning during class time is the other necessary feature of the classroom flip (the arrow going to the right and downward). These two foci influence student learning environments in fundamental ways (the two arrows coming together at the bottom of the diagram). These core ideas drive the conceptual framework for this research, and the chapter is sectioned to correspond with this structure. p. 28

As university and college faculty were introduced to CMS on a large scale in the late 1990’s and early 2000’s, many professors began changing the way they plan learning in the classroom (Collins, de Boer, & van der Veen, 2001; Collins et al., 2002; Hopkins, 2002; Lage et al., 2000; Phillips, 2002; Schauf, 2002). p. 31

Talks about how different technological innovations impacted the learning space and pedagogy, blackboard, textbooks, CMS etc. A nice way of framing it - like Jim talked about the introduction of new media, McLuhan etc. p. 31

As faculty explored how CMS could be used to encourage collaborative learning, group projects, and increased communication within the class, Wes Baker and Maureen Lage, Glenn Platt, and Michael Treglia independently saw the possibility of using CMS to completely invert the traditional structure of the classroom, moving the lecture outside the classroom and active learning opportunities into the classroom. Baker (2000) called his move the “classroom flip.” Lage, Platt, and Treglia (2000) called it the “inverted classroom,” stating, “Inverting the classroom means that events that have traditionally taken place inside the classroom now take place outside the classroom and vice versa” (p.32). p. 31

These seem to be the foundational texts, read! p. 31

Considering these developments in context shows that the classroom flip idea has been driven more by praxis than theory, and it appears this trend will continue. Workshops are offered that encourage and equip university and college faculty to employ p. 31

the classroom flip (Baker & Mentch, 2000; Bowers, 2002; OCTET, 2003; Ursuline College, 2002). In their 2003 strategic plan, Roane State Community College encouraged faculty to try inverting their classrooms (Roane State Community College, 2003), and a textbook company is solely in the business of selling (paperless) video lecture textbooks (Thinkwell, 2002). Obviously, the practical push for increased use of the classroom flip is strong. It is this practical push that sparked this investigation. p. 32

Intelligent Tutoring Systems While professors employing the classroom flip have usually relied on web-based course management systems to help students learn course content outside the classroom, I will use an intelligent tutoring system (ITS) to fulfill this purpose. This section describes the development and use of ITS over its short history. Early computer use in education resulted in what has been termed Computer Assisted Instruction (CAI). As cognitive psychologists, educators, and computer programmers (particularly those in the Artificial Intelligence [AI] field) looked more closely at what a computer can (and should) do to help students learn, ITS emerged with the goal that one day computers will tutor students with the same skill that human tutors do (Lesh & Kelly, 1996). Early ITS From the 1950’s through the 1970’s, many workers in the AI field were full of optimism that a computer could be designed to think like a human (Urban-Lurain, 1996). Scientists such as Alan Turing felt the limitations were mainly with computer power and that within the next 10 years, technology would be developed which was powerful enough to build a “thinking” computer. Unfortunately, the goal always seemed to be 10 p. 32

years away; despite growing computational power, certain entrenched problems kept the goal out of reach. During this same period, the first CAI programs were developed. These early programs presented information to learners in a linear fashion, essentially making themselves computerized flash card systems that kept track of correct and incorrect student responses (Urban-Lurain, 1996). Although these programs helped students perform basic skills and provided personalized instruction at the student’s own pace, they were plagued with serious limitations. Because of the linear sequence of instruction, these programs tended to promote a passive approach to learning. Students were not able to use natural language to solve problems and were forced to learn in a multiple-choice environment. This caused students to see the learning program as rigid, which in turn stifled motivation, initiative, and creativity (Jamieson, 1991). p. 33

Eventually, programmers developed CAI programs that responded differently to the learner depending on how they answered different questions. For example, if a student answered a question incorrectly, the program would give a response specific to that incorrect answer and then offer a remediation problem to try and correct the learner’s mistake. The program would be designed to go in a different direction depending on which incorrect answer was chosen. These programs were termed “adaptive CAI” or “frame-oriented CAI” and were the precursors to ITS. Although adaptive CAI were the first programs to begin to model student behavior, they still could not escape the limitations of earlier CAI programs. When using an adaptive CAI program students should always, perhaps after a number of programmed detours because of incorrect responses, reach the point were they were supposed to have mastered the material. If p. 33

students did not master the material, however, at some point they were bound to circle back to a set of questions they had previously seen and did not understand (Lelouche, 1998). In essence, adaptive CAI suffered from the same linear limitations as their predecessors, albeit after a number of possible detours. p. 34

What makes ITS different from adaptive CAI is that ITS actually conducts experiments with the student to help the system decide the content and teaching strategies needed for a specific learning session. To accomplish this, the system must have its own diagnostic capabilities and problem solving expertise, making it possible for the ITS to give effective instructional advice to the learner (Sleeman & Brown, 1982). It is not easy to describe all that is involved in building a system with these diagnostic capabilities, but most ITS developers agree an ITS consists of four components: the expert module, the student model, the tutorial module, and the evaluation module (or user interface) (Albert & Schrepp, 1999; CoMPIO, 2001; Cruces & de Arriaga, 2000; Duchastel & Imbeau, 1998). p. 34

Expert Module While educators formerly dreamed of offering entire courses through CAI programs, ITSs were developed with the idea they would be used to teach a modularized curriculum. Because the system must operate like an expert human tutor, it must contain (and know how to use) all the knowledge of the topic at hand (or the domain). This principle exerts tremendous pressure on ITS developers to stick to specific (even specialized) domains lest the system grow to unmanageable proportions. If computer programmers were going to be successful in writing programs that could tutor students the way human tutors do, they would need to develop computer p. 34

systems that (1) contain “expert” knowledge and (2) know how to use that knowledge to solve problems. These types of computer systems began to appear, but in AI rather than in education. As AI researchers came to terms with the idea that they would never develop a thinking computer, they began to explore what became known as “expert systems” (i.e. computer systems that can solve problems the way human experts do). The first applications of expert systems were found in business. Another example includes systems that were developed to predict weather patterns for weather forecasters. These expert systems were able to take all relevant information into account and make decisions just as competently if not better than human experts (Mueller, 2001). p. 35

Student Model To be intelligent, an ITS must have some understanding of the student it is tutoring. Therefore, ITSs incorporate a module that gathers information on the student with regard to the student’s understanding of the domain, learning patterns, and personal preferences/learning styles. Most early ITSs tended to focus on modeling the student’s knowledge but gave very little attention to the student’s personal learning style. There are three different ways ITSs model student knowledge. First is the overlay model. This model views student knowledge as a subset of the expert knowledge within the system (Goldstein, 1982). The second model, called the differential model, focuses on the difference between the expert and student’s knowledge (Burton & Brown, 1982). The third model is the perturbation model, which characterizes incorrect student responses as misconceptions of expert knowledge (Burton, 1982). However an ITS models student knowledge, an important challenge for developers is that of taking student knowledge, learning patterns, and preferences into p. 35

consideration and adjusting the pedagogical style of the system to fit the individual student (Okamoto, Cristea, & Kayama, 2001). While this remains an ideal to strive for, there is much room for improvement. Wenger (1987) addressed these issues when he asserted that student models must gather information about the student, create a representation of the student’s knowledge and learning preferences, use this representation to predict the student’s action, and compare the student’s actual action to the prediction and refine the model of the student. An effective student model will have some way to assess student knowledge either by continual interaction or by a specific “test-like” situation. It is difficult to make reliable assessments, however, because students do not always respond consistently, even when they understand certain concepts. Good ITSs will account for this kind of “noisy” data. p. 36

Evaluation Module Since ITSs are integrated systems for learning, it is sometimes difficult to speak only of one aspect (or module) of an ITS. The evaluation module overlaps considerably with the student model (and the tutorial module), and it is the part of the ITS that evaluates (or assesses) student knowledge and preferences. What this evaluation looks like will depend considerably on how the ITS models the student. Evaluations can include traditional tests and surveys or continual evaluation of students based on their overall interaction with the ITS. p. 36

Tutorial Module The tutorial module of an ITS is the part that interacts with the student in order to teach new material. According to a report put out by the Consequence Management Program Integration Office (CoMPIO) of Carnegie Mellon University’s Learning p. 36

Systems Architecture Lab (LSAL), the approach to the tutorial module is where we find the largest source of variation among different ITSs (CoMPIO, 2001). Even within one ITS there may be many different learning theories at work. Some ITSs use sophisticated dialogical interaction with the student, while others are problem driven. Whatever the case, the tutorial module will be largely influenced by the subject matter and the educational philosophy (whether implicit or explicit) of the developers. p. 37

Examples of Early ITSs p. 37

Carbonell developed the ITS called SCHOLAR in the early 1970’s to teach the geography of America (Duchastel, Doublait, & Imbeau, 1988). This system used Socratic dialogue as the pedagogical tool. SCHOLAR not only asks questions of the students, but it also allows students to ask it questions. p. 37

Notice that SCHOLAR is instructible in the sense that someone other than the ITS developer can add to the knowledge base of the system. This is a positive characteristic in p. 37

that the system can change as the world changes, but it is a negative characteristic because students (or teachers) could enter incorrect information, causing the system to lose accuracy (Duchastel et al., 1988). p. 38

Dialogical pedagogy was extensively explored in a medical ITS called GUIDON (Clancey, 1982). GUIDON’s goal was to build a tutorial system on top of the medical knowledge base/expert system on infectious disease called MYCIN. GUIDON is considered a “case method tutorial program” because it interacts with students by presenting them with case studies on different patients and then enters into dialogue with the student to help them make the proper diagnosis. The rules GUIDON uses to guide the dialogue are very complex and are meant to closely mimic the dialogue of a human tutor. For example, GUIDON gives students help based on previous student questions and relevant information that the student does not yet seem to understand. GUIDON can also use creative techniques like “entrapment” to force students to make a decision that will reveal their understanding of the material, as well as possible misconceptions they may have. This is a technique human tutors often employ. p. 38

Another important ITS, called SOPHIE, was developed by Brown, Burton, and de Kleer in 1975 to help The U.S. Air Force train advanced electronics engineers about complex circuits (Brown, Burton, & de Kleer, 1982). SOPHIE went through three major revisions, and in the process contributed invaluable information to the ITS field regarding pedagogical techniques, natural language engineering, and knowledge engineering techniques. Although SOPHIE has been highly criticized for not constructing humanoriented explanations, it provided a “jumping off point” for future ITS projects. p. 38

Richard Burton developed another oft cited ITS called DEBUGGY which helped identify student misconceptions (i.e. “bugs” or “mal-rules”) regarding the rules of subtraction (Burton, 1982). The DEBUGGY diagnostics followed a perturbation student model, and provided the framework for many other ITSs that used similar student models. p. 39

The development of LISPITS was extremely important because it was the first ITS to be carefully developed with one specific learning theory as its foundation. John Anderson built the LISP tutor to teach the programming language LISP using his ACT* learning theory. LISPITS used “model tracing” to analyze where students’ knowledge diverges from the expert model and then provides hints to help get students back on track (CoMPIO, 2001; Merrill, Reiser, Ranney, & Trafton, 1992). p. 39

Criticisms Early criticism of ITSs came from Rosenberg (1987) who charged that the systems were grounded more on the opinions of system developers than on substantial learning theory. Winne (1989) agreed that learning theory needs to guide designs for ITS. Specifically, he suggested extending Sternberg’s theory of intelligence to ITS by having ITS (1) adapt to the student, (2) select different interfaces for instruction, and (3) give opportunity for the student to shape the instruction. From a mathematics education perspective, Thompson (1989) and Lesh and Kelly (1996) offer serious criticism of the development of traditional ITS. Thompson hypothetically states that if he were interviewing a potential teacher and she tutored students the way many ITS do (with heavy-handedness and lack of perspective), he p. 39

would not hire her. Lesh and Kelly provide further criticism by comparing ITS and successful human tutors. They describe ITS-like behavior as: p. 40

• determining the exact nature of student knowledge
• comparing novice and expert knowledge states
• asking sequences of small questions to move students toward a pre-specified goal
• giving students immediate feedback regarding the correctness of their responses
• providing hints and helps if a student fails to learn. (Lesh & Kelly, 1996, p.151) p. 40

Lesh and Kelly’s research established that successful human tutors: p. 40

• spent relatively little time diagnosing students’ procedural bugs
• emphasized multiple linked representations as powerful instructional devices
• encouraged students’ representational fluency
• offered “fuzzy” questions to students that were self-adjusting in difficulty
• encouraged students to use technology to move past procedural details
• focused on following the students’ thought processes rather students following the tutor’ s
• often ignored student errors or purposefully induced them to confront student misconceptions
• modeled knowledge as deep understanding of an elementary topic area rather than as expert and novice knowledge states. (Lesh & Kelly, 1996, p.152) p. 40

Contrasting these two lists suggests that human tutors are least effective when exhibiting ITS-like behavior and most effective when exhibiting non-ITS-like behavior. p. 40

Thompson’s strongest criticism of ITS is that there seems to be no teacher in the picture. In his view, software should be developed to support classroom teachers with the assumption that teachers will use the software to help students learn. Thompson also gives a convincing argument against a rule-based ITS like the DEBUGGY system. These types of systems encourage students to imitate some observed behavior (like how to subtract with abstract symbols) only to analyze errors made by those students who are merely marking on paper and, in effect, reasoning without meaning. Rather than defining competence as possession of correct rules, Thompson contends that computer systems p. 40

should encourage proper conceptualizations of mathematics through teaching relationships among quantities. p. 41

Lesh and Kelly (1996) call into question any ITS that models student knowledge by measuring the difference between student knowledge and expert knowledge because (1) there are no canonical novice and expert states, as such, when dealing with humans and (2) the development from a novice to an expert is not continuous. Beyond this, experts and novices may share the same knowledge, but they understand it very differently, and this difference is not additive. These conflicts present fundamental problems to the way ITS traditionally model the expert domain and students as well. They conclude, “we believe it is preferable for students to tell computers what to do and to evaluate the computer-generated results, rather than for the computer to tell students what to do and to evaluate the student-generated results” (Lesh & Kelly, 1996, p. 139). Current and Future ITS p. 41

The ITS field is currently growing, and the limitations of pedagogical approaches are loosening in creative ways. Following Winne’s (1989) recommendations, ITSs are beginning to adapt their instructional interactions differently for specific students. The most significant development has been the introduction of multiple agents (or characters) within the ITS. There has always been a human student and a computerized tutor interacting within an ITS, but now one or more computerized students are being introduced into the picture (Chou, Chan, & Lin, 2003). These students can take on competitive, collaborative, or troublemaker roles within the ITS. Some human students may prefer the challenge of working in an environment where they compete with another (computerized) student to see who knows the material more thoroughly. Other students p. 41

may prefer to work collaboratively with another student who needs their help. In these situations, a student can work to complete a task and then learn the material at a deeper level by teaching the computerized student parts of the content. In other situations, the computerized student may take on the role of a troublemaker who sometimes injects erroneous information that the human student needs to filter out in the process of completing a task (Aïmeur, Frasson, & Dufort, 2000). By introducing the human student to different types of (computerized) learning companions within the ITS, developers can provide a richer pedagogical environment for the student that more closely matches the student’s learning style. p. 42

Other breakthroughs in ITS are occurring with respect to accounting for the unpredictable behavior of students. It is important for a truly intelligent ITS to be able to deal with students’ careless errors and inconsistency while learning. By using the results of a knowledge assessment to refine the model of student knowledge, researchers can allow students to act unreliably while working in an ITS and still recover an accurate model of students’ knowledge (Cosyn, 2002; Cosyn & Thiery, 2000). This important development has come from recent work in the field of knowledge space theory. ITS, Knowledge Spaces, and the Classroom Flip p. 42

Knowledge space theory is built on the premise that the domain for a given topic can be described using a formal mathematical structure without any reference to its interpretation in human minds (Lukas & Albert, 1999). I loosely relate the domain in knowledge space theory to cultural meaning as described by van Oers (1996). This will be developed in the next section. Each bit of knowledge in the domain is called an item. The theory contends that each student has mastered a certain number of items, and these p. 42

items are called the student’s knowledge state. Based on the student’s knowledge state, she will be ready to learn new items in the domain. These ready-to-learn items are called the outer-fringe (see Figure 2.2). Similarly, there may be items that the student understands but has not quite mastered; these items are called the inner fringe. According to knowledge space theory, as students strengthen their understanding of inner-fringe items, those items become part of their knowledge state, and as students work to learn new content, outer-fringe items move to the inner fringe. Working in this way, students will be able to progress through a learning path of ever growing knowledge states that eventually ends in mastery of the domain (Albert & Schrepp, 1999; Doignon, 1994; Doignon & Falmagne, 1999; Falmagne, 1993). p. 43

As part of the student and assessment modules, ALEKS determines all possible knowledge states within the domain by considering the prerequisite relationships between items in the domain. For example, it would be impossible for a student’s knowledge state p. 43

in basic arithmetic to only consist of multiplying two single digit numbers since a prerequisite for understanding multiplication is understanding addition. ALEKS then uses a sophisticated assessment tool to determine a student’s knowledge state. Based on the student’s knowledge state, ALEKS goes into a learning mode and presents the student with a new problem on his outer-fringe. The student can view different explanations of how to solve the new problem, or he can try to solve the problem right away. Then, ALEKS gives the student an opportunity to practice the problem until the item is mastered. ALEKS continues offering problems to students in this way until the entire domain is mastered. Periodically ALEKS will give the student subsequent assessments to determine which of the new items have actually made it into the student’s knowledge state. All through the learning and assessment modes, ALEKS does not use multiplechoice questions but provides students with the means to enter answers using standard notation. Students are also able to view their recent progress, see a graphical representation of their knowledge state, and print out personalized review worksheets. p. 44

In a flip classroom, teachers can use an ITS like ALEKS to introduce students to course content outside the classroom. ALEKS gives a full explanation of course content and provides examples of the concepts when students are ready to learn them. Well developed ITSs tend to have minimal technical difficulties and are designed to have a comprehensive knowledge base that should satisfy teachers’ requirements for content coverage. Because the ALEKS ITS meets both of these criteria, it was chosen as the technology system used in the classroom flip course for this study. p. 44

Classroom Flip → Learning Through Activity p. 45

The classroom flip is usually motivated by a desire to give students an opportunity to learn through active participation in the classroom. This motivation, however, needs clarification. What exactly is meant by active participation? Is not all learning active, whether learning from a book, a lecture, or a small group activity? Piaget says that learning occurs not when a person merely copies an idea, but when a person acts on it. When people really learn something it will be because they have developed a system of ways to (actively) transform the object of their thought (Piaget, 1971). Perhaps what we should be saying, then, is that the classroom flip provides opportunities for students to learn through a variety of different kinds of activity in the classroom. It is important, then, to have a framework for analyzing learning through activity. p. 45

A good place to start in building a framework for learning through activity is with a concept Bruner (1990) terms rebus, or understanding by doing something rather than just talking or thinking. The word rebus (Latin for “by things”) refers to how things rather than words can control our activity. Note that students can know how to do something using a tool long before they understand conceptually all that is involved. This happens frequently in the mathematics classroom. For example, students can know how to use a calculator to take a square root when they do not conceptually understand what a square root is. When asked, these students sometimes are not able to describe the difference between squaring a number and taking the square root of it. This example fits well with one of Bruner’s related assertions that instruments and aids often define work before it is completed. p. 45

Developing deep understanding, then, can involve using aids and instruments to complete an activity and later deciphering why it was necessary to complete the activity in that particular way. So, a barrier to learning occurs when students ask, “Now, why am I doing this?” only to continue their activity without finding an adequate answer to that question. Of course, students cannot be expected to completely understand the full scope of their activity up front, and learning theorists do agree that imitation is an important aspect of learning. However, Vygotsky contends that the focus of learning should be the development of higher psychological function, not just imitation. Through the use of language and signs (symbols that refer to thoughts and concepts), the learner is able to mediate their activity and move from a life where action dominates meaning (i.e. rebus) to one where meaning dominates action (Vygotsky, 1978). This movement, however, is a process. Students must be given a space where they can reflect on their own thinking and actions in the learning process to fully develop their understanding of classroom content. p. 46

In this manner, Dewey (1990) argues in his classic text The School and Society, that practical activity and reflection should guide learning: p. 46

Dewey’s comments hold Bruner’s rebus and Vygotsky’s higher functioning in healthy tension. It is not enough for students to do the activity; they must also get “the idea” of the activity. Getting the idea will necessarily involve guiding students to reflect on and assign meaning to their activity. It is important to note here that understanding does not p. 46

occur only at the end of activity; rather, students should form some kind of intellectual conception of the task at hand “from the start.” p. 47

Along these lines, van Oers (1996) has devised a theory of learning that informs the “give and take” between doing the activity and getting the idea of the activity (i.e., doing the mathematics and getting the idea of the mathematics). His theory assumes two types of meaning in the learning process: cultural and personal. Cultural meaning can be thought of as generalized knowledge. It refers to the concepts, norms, methods, and skills that have been built up within the mathematical community over the centuries. Because of its generalized state, cultural meaning can be converted into curriculum content, and taught. Personal meaning (or sense) is based on the value that learners attach to activity in relation to the task at hand. This personal sense develops as learners ascribe significance to the actions and goals of activity with regards to their motives, ambitions, and position in society. Undesirable rote learning is often the result of instruction that denies students opportunities to attach personal meaning to the instructional communication of cultural meaning. p. 47

Meaningful learning occurs in van Oers’ system as students develop both cultural and personal meaning in their educational setting. When students are able to master technical mathematical procedures as historically developed while also attaching personal meaning to the methods and results, they have achieved meaningful learning. How then can instruction be designed to foster this type of interaction between cultural and personal meaning? Van Oers argues that the link between both types of meaning can be reinforced through student reflection on the execution of their actions. He officially defines an action as “an attempt to change some (material or mental) object from its initial form into p. 47

another form” (van Oers, 1996, p.97). Therefore, as students act in the learning process, it is essential for them to reflect on their actions; van Oers calls this reflection orientation. p. 48

Through orientation, students are able to use cultural knowledge, rules, methods, and concepts in a personally meaningful way to complete the task at hand. Drawing from the classic Piagetian notion of reflective abstraction (Piaget, 1971), the orientation process results in students relating their actions to signs and symbols and eventually interiorizing them (transforming them to a mental level). Once interiorized, actions can be performed by either referring to their signs or just thinking about them. This is a very important step in the development of mathematical thinking. In fact, van Oers contends that the orientation (reflection) part of activity should be the fundamental focus of the teaching and learning process. From this perspective, mathematics instruction should primarily be concerned with guiding students’ orientational actions so they can explore and solve problems using culturally accepted methods. p. 48

One way to encourage reflective abstraction is by asking students to think of a mathematical concept using different representations of that same concept. Representations occur internally as constructs and schema in the student’s mind, and externally as words, symbols, graphs, charts, and pictures (Pape & Tchoshanov, 2001). Students often show that they have developed deep understanding of mathematical concepts when they are able to represent concepts using multiple representations and can freely move between those representations while thinking about a problem (Meyer, 2001). It is therefore crucial for the teacher to encourage students to use various representations when doing mathematical work, and then to have patience while analyzing student representations. Students sometimes use non-standard representations p. 48

to correctly represent mathematical concepts, and although their representations may appear incorrect to the accepted mathematical culture, a careful interpretation may provide valuable insight that can be used to guide students’ learning toward a deeper understanding of the mathematics involved. Teachers therefore act as negotiator/mediator between cultural meaning and personal meaning for the students by analyzing their representations. Because of this, mathematics teachers and researchers alike should primarily be interested in students’ evolving personal representations (Golden & Shteingold, 2001). Instructors assist students by offering suggestions, giving hints, openly scrutinizing, voicing objections, and encouraging alternate forms of representation. In other words, teachers enter into many different types of conversations with individual students and groups of students, and it is through these conversations that representations are used to negotiate, refine, and explain mathematical concepts (Pape & Tchoshanov, 2001). Without question, the key benefit of focusing on representations is that educators can see student knowledge construction as it unfolds, be involved in the learning process, and give feedback and guidance to students along the way. p. 49

Activity Theory p. 49

I will begin with Vygotsky’s dissatisfaction with the dominant behaviorist stimulus-response model of explaining human action (S R) in early 20th century psychology. He argued that signs (something as simple as tying a knot on your finger to remember to do something) change human psychological function and, as a result, render the stimulus-response model inadequate for describing most human behavior. The presence of signs in human psychological function allows humans to “control their behavior from the outside” and raises psychological operation to a qualitatively higher level where signs permit humans to mediate their behavior (responses). Vygotsky explains, p. 50

The use of signs leads humans to a specific structure of behavior that breaks away from biological development and creates new forms of a culturally-based psychological process. (Vygotsky, 1978, p.40) p. 50

The first generation of activity theorists adapted Vygotsky’s model to conceptualize a subject’s action upon some object (mental or physical) in order to achieve an outcome. They describe a subject’s action as being mediated internally by signs and externally by tools. Tools are more complex than signs and are aimed at controlling behavioral processes. Examples of tools include: language, number systems, works of art, literature, maps, and mechanical designs (Engeström, 1987). In the study of mathematics, we can view representations of mathematical concepts (graphs, equations, and linguistic explanations) as the tools and signs students use to mediate their learning activity. Recent research has shown how dynamic technological tools (like the Cabri geometry system) are not just used by learners to act on an object, but that these tools can actually change the ways learners construct and process knowledge (Rivera, 2005). I have adapted a common reformulation of Vygotsky’s model in Figure 2.4 (Engeström, 2001; Scanlon & Issroff, 2005). p. 51

This new way of thinking about activity allowed psychologists to move beyond the stimulus-response model toward a more contextualized understanding of human action; however, the first generation of activity theory’s focus on the individual as subject limited the scope of its influence. The subject’s community has a profound impact on his action, and thus needs a place in the model. The second generation of activity theory made room for community and conceived of the collective activity system depicted in p. 52

Let’s consider how a mathematics classroom operates as an activity system. One desired outcome is learning the material. If we consider the student as the subject and the p. 53

material to be learned the object, the tools and signs a student uses to act on the material will involve language, symbolic algebra systems, books, computers, heuristics, and study plans. The community where this activity takes place is the class itself, but it could also include tutors, apartment-mates, and anyone else with whom the student engages the material. Within this community, there will be rules that govern when certain mathematical techniques are preferred over others, how time should be used in the classroom, when students should come to class, and how students should complete their work. The division of labor in the community is distributed in different ways as the student acts on the material to be learned. The professor is expected to provide lectures or activities in the classroom that guide the student as she acts on the material. If the student is working in a small group, other members of the group may be expected to perform certain tasks while the student does something else. All of these elements in the activity system work together to help or hinder the student as she learns the material in the course. p. 54

Due to their interactive and dynamic nature, activity systems are continually driven to change by contradictions that arise between the elements within the system. For example, a student may see the object as “material I want to learn” or as “material I must get right on a test.” The same student may hold each of these views at different times throughout the semester, and balancing these competing/contradictory views will change the tools the student uses in the learning process as well as the rules used to engage the material. If a student wants to learn material because of some intrinsic motivation, he may use a heuristic that promotes long-term learning as a tool when acting on the material. However, if the student views the material as stuff that must be learned for a p. 54

test, he may use a heuristic similar to “cramming” that is more likely to produce shortterm learning. Rules that govern time allocation for learning will also most likely be different depending on which of these two views the subject has of the object. p. 55

Activity systems are also in a state of flux due to the movement between the elements of the system. Something that begins as an object in the system may later become a tool that needs a new set of rules to govern its use. For example, when a student first encounters a histogram in a statistics course, it is an object to be studied and acted upon. Once the student knows how to make a histogram, it moves from being an object to being a tool. The system then needs to adjust, and rules must be developed that tell the student when it is and is not appropriate to use a histogram as a tool when acting on other mathematical objects. The potential for movement between the elements of the activity system in this way are endless. An object may become an outcome, then a tool, and may eventually become a rule (Engeström, 2003). Rules may also be challenged, reconsidered, and changed. This sometimes causes them to become new tools and objects themselves along the way. p. 55

Activity theorists have conceptualized the messiness of activity systems by separately defining activity, action, and operation. Activity, when viewed from this perspective, becomes a (muddled) collection of actions carried out in the context of community. Actions themselves, taken one at a time, tend to be more direct. Relatively clearly defined goals guide actions, and they can be carried out by individuals or groups of people. Actions, in turn, are made up of lower level operations that began as actions, but have been interiorized and routinized to the extent that they can be performed almost as an automatic response to an observed condition. People or even machines can perform p. 55

operations. A graphic of this hierarchical nature of activity, action, and operation is p. 56

The bi-directional arrows in the figure show that activities, once worked out, can be compressed into actions. And actions, if repeated enough, can be automated to the point of becoming an operation. Conversely, if an operation is disrupted, it can be raised again to the level of an action, and an action may be raised to the activity level (Jonassen & Rohrer-Murphy, 1999). For example, learning how to create a histogram to display a numerical variable’s distribution may be an activity carried out as an entire class. The nuances of the meaning behind histograms are explored and students think deeply about each part of the display. Once the idea of a histogram has been internalized, students can begin to make histograms as only one action in the larger activity of solving a more complex problem. This pushes the creation of a histogram down to the level of an activity. Eventually, students may use a computer or other technology to create a p. 56

histogram. At that point, making a histogram becomes an operation. However, if there is a problem with the technology, or if students wish to compare two different variables using histograms, they may need to again engage the histogram creation as an action (adjust class widths, etc.) or even an activity (to consider how they should work with the histogram to decide if there is a significant difference between populations, for example). p. 57

The question that has driven the development of the third generation of activity theory is, “What happens when activity systems from different cultures and communities interact?” Since the second generation’s conceptualization of activity did not allow contributions from multiple communities, the third generation of activity theory has worked to develop ways to conceptualize the dialogue and interaction between different communities and traditions. Researchers are envisioning how networks of activity systems may operate and how people cross borders between systems to act on shared objects that may have different meaning in different systems. Gutiérrez and her colleagues (Gutiérrez , Baquedano-López, & Tejeda, 1999) describe how a “third space” is created when two activity systems interact, and that this third space may allow for the development of new meanings that go beyond the limitations of the two original systems. p. 57

Learning Through Activity → The Learning Environment Educational Technology → The Learning Environment p. 59

Learning environments research has its roots in a sociological study of human environments in general (Moos, 1973). Moos and Walberg independently applied human environments research to learning environments in the 1960s and 1970s in the USA. Since then, others have built on Moos and Walberg’s work, but the most extensive work has been done by Fraser and his colleagues in Australia (Dorman, 2002). Eventually the Learning Environments Research special interest group of the American Educational Research Association was formed and this group launched the Learning Environments Research journal in 1998. Operating from a solid theoretical base, learning environments research has been committed to the improvement of educational policy and classroom learning. p. 59

Perhaps the most important and enduring theoretical development in learning environments literature is Moos’ three domains of social-environmental variables. Moos (1979) asserts that vastly different social environments can be conceptualized using variables that fit into three broad categories: 1) relationship, 2) personal growth, and 3) p. 59

system maintenance and change. These three categories must be described and examined (at a minimum) if researchers wish to gain understanding of the social environment under investigation (Moos, 2003). p. 60

The relationship domain has to do with the extent to which students are involved in the classroom. The variables in this domain are closely tied to the sense of community students feel with one another and the teacher. They include the extent to which students help and support one another, the level of morale in the class, how freely students share ideas, how students share in work responsibilities, the level of participation in classroom discussions, how attentive students are to the course content and the class activities, and the depth of student relationships with other students and the teacher. p. 60

Personal growth dimensions involve how the goals of the classroom encourage student development and learning. Student independence is key in this domain since true learning and development occur when students are free to succeed or fail. The ways the environment encourages students to be aware of significant course concepts is also an important feature of this domain. If a classroom places a great deal of importance on academic, intellectual, and scholarly activities, the students’ desire to grow and learn will be affected. Other factors that influence personal growth and learning in a classroom setting include the level of competition in the class and how the grading policies in the course are structured. p. 60

The system maintenance and change domain is concerned with the formal structure of the classroom and how this plays out in the day-to-day operation of the classroom. The clarity of classroom expectations, the ways communication occurs in the classroom, the rules and policies that govern normal operation in the classroom, the p. 60

practicality of the classroom (how orderly it is supervised), classroom propriety (how considerate and polite the environment is), and the ways the classroom adapts to change all relate to the system maintenance and change domain. These dimensions set the stage for how the classroom is run and how students interact within its structure. p. 61

Along with the three social-environmental domains, influential early conceptualizations of human environments dealt with the interplay between the needs of the individual and the press of objects in the environment (Dorman, 2002). Needs are defined as factors in the individual that influence behavior, and the press of an object involves the ways the object can influence (positively or negatively) the attitudes, actions, and motivations of a person. It is often difficult to determine the press of an object because of the subjectivity involved. For instance, if a student believes failing a classroom quiz will cause her to fail the course, then the quiz has a certain press on the student even if, in reality, the quiz is only worth 1% of the overall grade. This way of thinking about learning environments has led researchers to see the importance of the distinction between students’ preferred learning environment and their actual learning environment, and instruments to measure learning environments were developed with this in mind. p. 61

As individual students interact with their learning environments, the socialenvironmental domains of the classroom come into contact with the personal system each student brings with them to the class (see Figure 2.8). As a student learns, the interaction between her social environment and personal system instigates a series of struggles and adaptations as she strives to learn in that environment. A student’s personal system includes age, gender, ability level, interests, values, attitudes, expectations, and coping p. 61

preferences. When a student is confronted with a learning experience, she will experience a need to change. Her personal system and social environment will influence how she appraises the situation cognitively as well as the actions she decides to make within that environment (activation). Once the student acts in the environment, she will go through cycles of adaptation and coping until equilibrium is re-established and change (learning) has occurred. p. 62

I used the framework in two specific ways. First, the framework helped me separate the related research literature in Chapter 3 into two main sections: mathematical representations and learning environments. Second, the conceptualization of activity theory and learning environments as set forth in this chapter provided a lens through which I fit together and analyzed the data collected in this study. The framework was at work implicitly as I coded and categorized the qualitative data. However, it is important for research to show how the conceptual framework was at work explicitly in the analysis of the data. Therefore, in the Student’s Take sections of Chapter 5, I explicitly show how activity theory and learning environments theory shaped my analysis in this report. p. 63

There have been few research studies that specifically investigate flipped, or inverted, classrooms as strictly defined. I will review these studies first. There are many studies that investigate environments where technologies are used as a tool to increase student involvement in classrooms, and I will review these studies in the second part of this section. p. 74

In a study using the classroom flip, Baker (2000) provided lecture notes on a web page, extended classroom discussions through online threaded discussion, and used online quizzes in two of his courses (Graphic Design for Interactive Multimedia and Communication in the Information Age). His aim was to achieve the following goals: reduce time spent on lecturing, focus on understanding and application, provide students with more control over their own learning, give students a sense of responsibility for their p. 74

learning, and give students an opportunity to learn from their peers. Baker’s action research project evidenced increased interactivity and collaboration in both courses when compared with other courses the students have taken. Students noted an increase in collaboration both in the classroom and out of the classroom (using technology). Students felt they received more personal attention due to the structure of the class, had more control over their learning, and were able to engage in critical thinking that explored the implications of their learning (Baker, 2000). p. 75

In a study by Lage, Platt, and Treglia (2000), Introduction to Microeconomics courses were modified by asking students to read assigned sections of the textbook and view either videotaped lectures or PowerPoint lectures with sound before coming to class. The first part of each class session involved answering questions, which usually lead to a mini-lecture lasting no more than 10 minutes. If there were no questions, there would be no lecture. The rest of the class time was spent in an experiment, lab, or group work that investigated the topic at hand. Lage’s and Platt’s goal for inverting the classroom was to give students opportunities to learn economics according to their individual learning style. Students could learn course content by choosing between reading the textbook, watching a traditional lecture, or viewing PowerPoint with sound. They could also combine or repeat these content delivery methods according to their individual preferences. Hands-on activities inside the classroom added further diversity to the available teaching and learning styles. This study of 80 introductory economics students showed positive student attitudes toward the inverted classroom. In fact, the evidence showed that students would prefer to have an inverted classroom rather than a traditional lecture class. The study also evidenced increased faculty-student interactions p. 75

One possibility of inverted classroom is to give students more choice over their learning style etc p. 75

and the development of student communication skills. Since the material in the course is presented in a number of different formats, it was shown that students’ learning preferences were better matched to course pedagogy (Lage & Platt, 2000). p. 76

Although the classroom flip terminology was not specifically stated, this instructional method was used in a study involving 16 graduate level research methods and statistics students to compare lecture-based versus computer-based methods for presenting students with course content (Frederickson, Reed, & Clifford, 2005). These researchers were interested in seeing if there were differences in the level of learning and student opinions of the learning environment between the two groups. The 16 students were randomly assigned to two different groups: one group spent an hour in a computer lab going through a web-page based presentation of the material while the other students spent the hour in a classroom listening to a lecture over the same material with overheads and handouts. The random assignment in this study is a very important and unusual aspect when compared to other current studies. Most studies that compare different pedagogical approaches allow students to choose which group (lecture-based or technology-based) they will be in for obvious ethical reasons. However, in this study, all 16 students agreed to be randomly assigned to a group. It is also important to note that the information in the web-page presentation and the lectures were developed by the same instructor and were virtually identical. p. 76

Frederickson et al (2005) gave both groups of students pretests and posttests to detect changes in their statistical knowledge and levels of math anxiety. Results showed that both the lecture-based and the web-based groups increased their understanding of statistical knowledge from pretest to posttest. However, there was no significant p. 76

difference between the two groups on either the pretest or posttest. There were no other significant effects between the two groups in terms of their pre-anxiety and post-anxiety levels. The researchers also solicited written feedback and they performed a cursory qualitative theme analysis on those responses. p. 77

Frederickson et al.’s (2005) important study provides evidence that similar gains in knowledge will occur for students whether the material is presented in a web-based or lecture-based format, and math anxiety does not appear to be influenced by one method or the other. Qualitative data were also collected through open-ended questions on a survey given to all students. The analysis of this data suggested students were more critical of the web-based format. Students in this environment wanted their learning goals to be more clearly defined so they could check to see if they were “on the right track” along the way. They wanted more explanations and examples on the websites and a more interactive experience. Although the lecture-based students received the same stated learning goals, explanations, and examples, they made no mention of the need for more feedback and reinforcement as the web-based students did. p. 77

Analyzing these results (Frederickson et al., 2005) from an activity theory perspective suggests that the introduction of a new tool (the web-page driven learning modules) caused a disruption in the students’ activity system. Now, new rules were needed to define how learning was to occur in this environment. Further, the division of labor changed since the teacher was less involved in the presentation of material and the student had more responsibility. Students responded to these disruptions by taking more responsibility for deepening and monitoring their learning (the demand for more examples and clarity in learning goals). From a learning environments perspective, what p. 77

at first appears as student dissatisfaction with the system maintenance and change domain of the learning environment (unclear goals) may have actually had a positive effect on the personal growth domain since the environment now encouraged students to be more aware of the content and their own learning process. p. 78

One study has been conducted that investigated the learning environment of students who used the ALEKS intelligent tutoring system (Canfield, 2001). Three classes of 10 students each participated in a Basic Mathematics course that used ALEKS at a U.S. university and completed a questionnaire at the end of the term. Results showed that students liked the detailed explanations and feedback, the tailored review problems, and the self-paced nature of the work. Students also reported lower stress levels as compared to traditional lecture style mathematics courses they had taken. Eighty percent of the students in these courses reported that they learned as much or more in the ALEKS course as compared to other courses, they would take another mathematics course that used ALEKS, and they would recommend ALEKS to another student. Canfield contends that since ALEKS teaches the standard factual knowledge usually found in traditional lectures, teachers have an opportunity to make their classrooms a place where inventing, abstracting, conjecturing, proving, and applying mathematics in realistic situations is the norm. This is the essence of the classroom flip. p. 78

Technology-Rich Active Learning Environments p. 78

Broad, Matthews, and McDonald conducted a study to investigate the effect a virtual learning environment had on students’ learning preferences (Broad, Matthews, & McDonald, 2004). These researchers took content from an accounting course and packaged it in a hypermedia environment on a CD (similar to a courseware management p. 78

system like Blackboard, but self-contained). The content on the CD took many different forms (static text, hypertext, quizzes, PowerPoint presentations, practice exercises, and online links) so that a variety of learning styles could be accommodated. Students began working through this material at the beginning of the term and attended 2 lecture sessions a week as the term progressed. The lectures were meant to add value to the virtual learning environment through personal interaction and the ability to target the lectures at trouble spots for students. As students worked in this environment, researchers collected data on their learning preferences. p. 79

The significant findings of the Broad et al. (2004) study show that students using the integrated virtual learning environment became progressively less pragmatic in their approach to learning. Pragmatic learners tend to focus on doing what is necessary to complete assignments and are not as concerned with engaging in theoretical discussions or exploring the implications of the concepts they are learning. This suggests students have adjusted their approach to learning and as a result of the change in learning environment. This evidence is in accordance with the Frederickson et al. (2005) study suggesting that as students use courseware learning systems to learn content, they tend to adjust their activity systems in a way that strengthens their awareness of the learning process and course content (the personal growth domain). p. 79

A study involving computer science students at the university level studied learner preferences and students’ choice of learning environment (face-to-face or on-line) (Buerck, Malmstrom, & Peppers, 2003). Twenty-nine students at a U.S. university participated; all were working at least 40 hours per week and all were at least 22 years old. Results showed that there was no significant difference in the academic performance p. 79

of the two groups. There was evidence, however, that computer science students who chose the on-line format were converging learners and students who chose face-to-face were assimilating learners. Convergers tend to be good problem solvers and decision makers. They more easily see practical applications for theories, and therefore prefer to work on technical tasks more than interpersonal ones. Convergers like to experiment with new ideas and therefore prefer laboratory assignments and practical applications. Assimilators, on the other hand, are good at transforming information in a logical manner to create theoretical models. Therefore, assimilators prefer to read, hear explanations, and explore analytic models. Having time to think through things is very important for assimilators. p. 80

Converging vs assimilating learners - different approaches suitable for different kinds of learners? p. 80

While Buerck et al. (2003) looked only at computer science majors, their study speaks to all fields because it highlights an aspect of the personal system that students bring with them into the learning environment, learning preference. Students with different learning preferences will adjust differently to a technologically rich learning environment, some needing more time and help than others. If possible, the technological learning environment should be developed so that learners of all different styles can find ways to learn that are comfortable for them (see Lage et al., 2000). p. 80

In a study that investigated the influence computers had on middle school social studies classrooms, Mucherah (2003) noted the important ways in which the learning environment was affected. Three hundred and six students from 14 classrooms in 3 urban middle schools on the east coast of the U.S. participated in this study. Data were collected in this study through student questionnaires that measured perceptions of the classroom environment, graduate student observations of classrooms, and interviews with p. 80

participating teachers. Evidence from this study suggested that while students’ perception of the classroom environment explains a significant amount of variation in student learning, there is often a disparity between how students perceive the classroom and how teachers perceive it. Teachers in this study believed that when students learned with computers they tended to stay on task better, work more cooperatively, and be more motivated to learn. These teacher beliefs contributed to the perception that students interacted in the classroom and were involved in class activities while the teachers provided support for learning. p. 81

Student perceptions of the environment painted a different picture, however. Students tended to see the classroom as a rule oriented place, highly controlled by the teacher with activities structured by the teacher. This result demonstrates how the classroom activity system can look very different depending on who the subject is. The dynamic dependence of activity systems on the subject is supported by previous research showing that people in positions of authority with more responsibility for the learning environment tend to view the environment more positively (Mucherah, 2003). p. 81

Beliefs about learning was the focus of another learning environments study that concentrated on changes in students’ instructional and epistemological beliefs as they participated in a problem-based, collaborative, and technology-rich project (Elen & Clarebout, 2001). This study measured the instructional and epistemological beliefs of 139 secondary level students from 3 different European countries. After the initial survey, these students worked for 4 hours a week for 8 weeks collaboratively to prepare a position paper for a member of the European Parliament on the increasing international mobility of the labor market. Students from each school were matched with students from p. 81

the other participating schools and required to solve the problem as a group. Students had e-mail, videoconferencing, telephone, fax, Internet, and a dedicated database at their disposal as they worked on the task. After the task was completed, students’ instructional and epistemological beliefs were again measured. p. 82

Elen and Clarebout (2001) defined instructional beliefs as the beliefs students hold regarding the optimal conditions for learning. For instance, if students think an activity is easy, they tend to put less effort into their work, which in turn influences their ability to engage the material and learn effectively. The second type of belief, epistemological beliefs, is the belief students hold about the nature knowledge. Examples include the view that knowledge as fixed and innate, or that knowledge as something built up over time (both in society and the individual). Evidence in this study showed that as students worked in this learning environment over 8 weeks, they significantly shifted both their instructional and epistemological beliefs. p. 82

The surprising result is that student beliefs about the benefits of learning through collaboration while using a variety of technological tools decreased. This result led the researchers to evaluate the learning environment as planned and the learning environment as it was implemented. They found that the implementation did not mirror the planned learning environment, and students’ changes in beliefs reflected their ability to adapt to their learning environment as it was implemented. This evidence further supports claims that the introduction of tools (assorted technologies) and objects (an open-ended collaborative assignment) into the learning environment can cause profound disequilibrium in the activity system. This disequilibrium will result in a need for a readjustment in the rules, division of labor, and the community itself. Without a concerted p. 82

focus on helping students through this re-adjustment, they can become disillusioned and withdraw from the learning process. p. 83

A study involving 406 students in a traditional assignment-based learning environment and 312 students in a redesigned problem-based learning environment investigated the influence of the environment on student level of learning (Nijhuis, Segers, & Gijselaers, 2005). The level of learning was characterized as being either deep learning or surface learning. Deep learning occurred when students showed interest in and searched for meaning in the learning task as they worked to integrate the individual parts of the task into a meaningful whole. Surface learning occurred when students only engaged the content enough to get the questions on the task correct. Students who only develop surface learning spend more time memorizing and reproducing information. They do not seek out further connections, meanings, or implications of the information learned. p. 83

Deep learning vs surface learning p. 83

Nijhuis (2005) used the Course Experiences Questionnaire and the Study Process Questionnaire to assess the level of learning for students in the two different learning environments. The researchers expected the problem-based learning students to adopt more of a deep learning approach than those in the traditional assignments based learning environment. However, the evidence showed that the problem-based learning students showed significantly more surface learning and significantly less deep learning than the assignments based students. An analysis of the factors that contributed to these results led the researchers to conclude that communication in the problem-based environment needed to increase. Teachers needed to communicate the ideas behind problem-based learning and give students more feedback as they process the learning tasks. Researchers p. 83

also concluded that students needed training to help them deal with the changes that the problem-based environment would bring to the classroom. These conclusions agree with Elen and Clarebout’s (2001) assessment of the complexity that changes in the learning environment bring to the community of students and their teacher. Again, these results are in agreement with an activity theory assessment of the fluid movement of the elements in a changing activity system. p. 84

Other studies have shown the potential of the classroom flip course design. When given more freedom to act in the classroom, students are often willing to step up if the instructor provides structural support within the activity system. However, if the classroom environment is not managed to handle the environmental changes, students’ learning may suffer. This is why it is so important for teachers to be aware of the domains of the learning environment and to provide a balance between them so that a healthy learning environment results (Moos, 2003). p. 84

A number of validated quantitative instruments have been developed to study learning environments. Since these instruments have driven most of the research on learning environments, it was important to use one of these to investigate students’ perceptions of their learning environments in this research study (Fraser, 1998; Fraser, Treagust, & Dennis, 1986). However, the complexities that make up the learning environment cannot be sufficiently accounted for by giving students a survey on one particular day towards the end of the semester. For this reason, many learning environments studies are incorporating multiple methods to investigate the intricacies of this research domain (Fraser, 1998). p. 85

I collected and analyzed data from student interviews and focus groups, in-class observations, audiotaped classroom sessions, student assignments, student written reflections, and researcher reflections. p. 86

One statistics class was structured according to the classroom flip method and met in a computer lab. Outside of class, students were introduced to new content by working with the Assessment and LEarning in Knowledge Spaces (ALEKS) intelligent tutoring system (for an explanation of all acronyms, see Appendix A). When students came to class, they completed activities that were designed to help them engage the content they p. 86

were learning in ALEKS in a different context. Students could interact with each other and the professor in class as they worked to strengthen their understanding of the more formal mathematical material presented in ALEKS. Often, these activities required students to use the Microsoft Excel spreadsheet program as a tool. p. 87

The other statistics class was structured according to a traditional lecturehomework format where students came every day to a classroom with tables and chairs and heard a lecture over statistics content. These lectures were heavily content driven. I would introduce statistical concepts and then work though examples that used those concepts. During the lectures, students had opportunities to ask questions or answer my questions related to the examples discussed. In this way, I made an effort to make the lectures as interactive as possible. After 2 or 3 class periods, students were assigned a set of problems from the book to complete as homework. p. 87

Most students in both sections agreed to participate in this research. Twenty-seven of the 28 students in the lecture-homework classroom and 23 of the 27 students in the flip classroom agreed to participate. One lecture-homework participant dropped the class a few weeks into the semester for scheduling reasons, so there were 26 participants who finished the study in that class. Participants from both classes were evenly split by gender (13-F, 13-M and 12-F, 11-M). The majority of students in both sections were underclassmen: 21 in the lecture-homework class, and 14 in the flip class. p. 88

Invited other people to observe class room and take notes, do some interviews because he was teaching it himself p. 88

I also was a part of the data collection team. I audiotaped class sessions, kept a reflective journal, took observations after class sessions using a course log, and conducted member checking interviews. In the paragraphs below, I describe my activities as an educator and researcher over the past 10 years or so. p. 89

I wanted this research to provide insight into how technology use and the classroom flip structure influence the learning environment for students and the instructor. p. 90

Research question focuses on learning environment - how technology and flipping influences it for students and professor. p. 90

First taught a course traditionally with ALEKS (tutoring system) as textbook to work out any glitches, then used it as flipped. p. 91

With two weeks left in the semester, students had experienced the flip environment and the lecture-homework environment for many weeks. This is the time I chose to administer the College and University Classroom Environment Inventory (CUCEI) questionnaire (Fraser et al., 1986). The CUCEI questionnaire provided insight into: (1) students’ perceptions of their actual learning environment and (2) students’ opinions of what their ideal (preferred) learning environment would look like. The CUCEI was developed to measure student and teacher perceptions of classroom psychosocial environment in college and university classrooms. The instrument is grounded in Moos’ theory that all human environments contain, at minimum, relationship dimensions, personal development dimensions, and system maintenance and system change dimensions (Moos, 1974). Pertaining to the relationship dimension, the CUCEI focuses on identifying the nature and intensity of personal relationships, assessing the extent to which students are invested in their environment and support and help each other. The CUCEI also seeks to assess the extent to which the environment pushes students toward personal growth and self-enhancement (the personal development dimension). Finally, the CUCEI strives to measure the overall orderliness of the environment, its responsiveness to change, and the clarity of expectations (the system maintenance and system change dimension). These dimensions of the environment, then, are measured on the CUCEI using the following seven scales: personalization, innovation, student cohesion, task orientation, cooperation, individualization, and equity. The development of this instrument was guided by findings of studies that used similar validated instruments to measure learning environments in elementary and secondary p. 92

schools. Further, the CUCEI’s internal consistency for the seven subscales has been measured in multiple studies and has been shown to be quite acceptable with Cronbach’s alpha coefficients ranging from 0.70 to 0.90 (Fraser, 1998; Fraser et al., 1986). p. 93

In this project, I employed an educational anthropology methodology for collecting data. p. 93

The methods I employed in this inquiry are termed ethnographic. p. 94

The ethnographic approach fit well with my research since the participant group under study (an Introduction to Statistics class) can clearly be viewed as a specific social group. p. 94

My research methodology placed me as both an insider (course instructor) and an outsider (educational researcher). These dual roles placed me as a participant observer throughout the research process. With this in mind, I needed to use data collection techniques that helped garner appropriate information. It was important for me to gather data that documented my thoughts and actions as instructor in order to get a fuller understanding of the context and movement of the course throughout the semester. Consistent with methods that include the researcher as a participant in the study (Ellis & p. 94

Bochner, 2000), I collected data by writing in a reflective teacher journal after specified class sessions. In this journal, I focused more on myself as teacher in the classroom. I documented my thoughts on how I thought class was going in general, my struggles and successes, the emotions I felt, how well I thought students were learning, and how I changed course instruction throughout the semester. This movement of looking inward and then outward between then the personal and the cultural is typical of this type of research. p. 95

I kept a “field notes log” that focused on student behavior in the classroom. I also wore a microphone to audiotape selected class sessions (at the beginning, middle, and end of the semester). After class was over, I listened to these tapes and wrote my observations of the class. I also had members of my data collection team come into the classroom and observe the classroom settings during the middle of the semester and then again towards the end of the semester. p. 95

Some literature references around observation and ethnography, participant observer, etc p. 95

In addition to observations and interviews, I looked to a variety of documents for information in this inquiry. I included documents created by students in the class such as reflective papers, projects, and exams as part of my data corpus. Wolcott (1995) argues that these varied types of data sources can often be critical to gaining deeper understanding of the phenomenon under study. p. 99

I started coding the qualitative data, moved into the quantitative analysis, and then finished the qualitative analysis. I chose this approach to mixed methods analysis in an effort to let the data speak for themselves and to minimize any initial bias that the quantitative survey could bring into the study. p. 99

I began analyzing my body of data by first open coding the qualitative data. After writing a number of exploratory memos on the unfolding analysis and coding a little over half of my qualitative documents (including student reflections, focus group observations, interview observations, exploratory memos, classroom observations, and transcripts of class sessions), I analyzed the College and University Classroom Environment Inventory (CUCEI) results using a number of quantitative methods. p. 99

Data analysis with qualitative data usually involves multiple phases that can occur simultaneously. These phases include organizing the data, generating categories and themes (through open coding, axial coding and memo writing), testing hypotheses (through axial coding, selective coding and memo writing), searching for alternative explanations (through axial and selective coding), and writing the report (Marshall & Rossman, 1999; Strauss & Corbin, 1990). p. 101

While I do not attempt to provide a point-by-point analysis of the concepts that underlie the traditional standards of validity and how qualitative research can meet these standards (this has been done elsewhere (Corbin & Strauss, 1990; Guba & Lincoln, 1981; Guba & Lincoln, 1985)) p. 108

Summary of CUCEI inventory, students stronger on preferred than actual, flipped students higher on creativity and collab, but miss task orientation. p. 119

These results present a dynamic flip environment where students prefer collaboration and innovative teaching strategies, but they are less satisfied with how the environment orients them toward course content. p. 120

An analysis of these data revealed that the two different statistics classes (flipped and traditional) developed two distinct classroom cultures. p. 121

The instructor’s reconstructions that follow were pieced together based on transcripts of the class sessions, log entries, reflective journals, and researcher memos p. 121

The reconstructions from the students’ perspectives were built based on the focus groups, interviews, and student reflection papers on their learning in the course p. 121

An important dynamic developed during the Jack investigation that persisted throughout the entire semester. Some students would work for a little while on the assigned problem, and then if they got stuck or frustrated, they would begin playing games or doing other work on their computers p. 123

Here is how we worked in the ALEKS system: I assigned new objectives every two weeks that covered the material we were also working on in class, and I encouraged students to work on ALEKS a little bit everyday. Next, I assigned follow-up assessments after every two objectives. These assessments started at the foundation and asked students a series of questions to see how well they had learned both old and new items. If students answered questions incorrectly during the assessment, even though they had completed the item in a past objective, then after the assessment was over, students were required to complete these items again before they could move on to the new material for the next objective. Because of this caveat, I tried to structure the in-class investigations so that students could either (a) initially investigate these concepts and successfully work through them or (b) if they had already solved these types of problems in ALEKS, they could have an opportunity to successfully transfer that knowledge to this new context. Eventually, all students completed the same ALEKS items and in-class investigations. p. 125

Organization of class: students used intelligent tutor at home, in class did project around a case, first 3 weeks, then worked on problems, then finally individual projects on the same case + exam. p. 130

Brenita Let's pretend I'm a friend from high school who will be taking a class from Prof. Strayer next semester. What would you tell me to expect from his statistics class in general? p. 132

Through the process of memo writing and constantly revisiting the original data and original codes (often recoding data under new codes), I was able to determine the following major categories in the data: classroom relations, logistics of class, theoretical influences on learning, practical influences on learning, personal/emotional influences, and classroom peculiarities. p. 167

All of this varied activity influenced the culture of the classroom so that students never really settled into a pattern for “how to do class.” At times, students clearly did not know what to expect or where class was going. p. 168

. For instance if a student approaches an activity wanting to be shown exactly what to do along the way, but the activity structure is open ended, then the student will be more likely to be frustrated than if the activity structure were step by step. The same goes for the other direction. If a student approaches an activity wanting to be allowed to struggle through and come up with creative solutions, but the activity structure is step by step, then the student will be more likely to be frustrated than if the activity structure were open ended p. 179

Students in the flip environment, on the other hand, had less freedom to act in the classroom. The structure of the activity to be completed (whether Jack or a smaller 1⁄2- sheet investigation) was set by the professor before class, and students were expected to actively engage with it during class. If students’ approach to the activity did not fit well with the structure of the activity, this caused disequilibrium for the student. In this environment, students had to negotiate the space between the structure of the activity, their approach to the activity, and their mind-set towards the activity. Sometimes this negotiation worked well and students progressed successfully through the activity. Other times, students failed to make it through and they struggled to make sense not only of the specific activity, but also of the direction of the class in general. p. 181

In the flip classroom, it p. 181

was not enough for students to engage the material only with their attention; they also needed to participate in classroom activity for class to go smoothly. p. 182

, the environment was organized in such a way that it made it disruptive for students to do other things during class like study for other classes, read other material (in a book or on the Internet), or just complete their homework (ALEKS). p. 182

I spent considerably more energy trying to manage students’ learning activity during class compared with the traditional class. p. 182

The decreased belief in the usefulness of collaboration was surprising for Elen and Clarebout. Due to the review of literature that they conducted, they were sure that the project would produce positive results with regard to beliefs toward collaboration. The apparent conflict led Elen and Clarebout (2001) to conclude there was a difference in the learning environment as planned and the learning environment as implemented. p. 200

The core category of comfortability with activity as developed in this analysis bears some resemblance to Bandura’s (1997) concept of self-efficacy. p. 202

When considering self-efficacy, it is important to distinguish it from self-esteem and academic ability. p. 203

The model for student comfortability with activity developed in this research fits well into Moos’ (1979) interaction model. The activity itself will provide the spark that sets the interaction in motion, and this activity will have a specific structure (ranging from step by step to open ended). Students will have a preferred approach to the activity p. 205

(ranging from “I want to you to show me” to “I only want to ask questions when I get stuck”) as well as a mind-set toward the activity (ranging from “I will do what I have to” to “I want to understand the purpose behind this activity”). Both the approach and the mind-set characteristics fall under the cognitive appraisal stage of Moos’ interaction model, and they will influence how the student responds to the activity at the “activation or arousal” stage. As students work through the activity, they will encounter disequilibrium (borrowing from Piaget) in their personal systems and make efforts to adapt and cope with the learning activity to eventually reach a place of stability and change. p. 206

An adapted model for the interaction between the personal and social systems of students as they complete classroom activity in light of the structure of the classroom is presented in Figure 6.3. Note how the properties of students’ comfortability with learning activity interacts with “cognitive appraisal” and “activation or arousal” and how the classroom structure will either support or hinder “efforts at adaptation and coping” and “student stability and change.” The classroom structure will support students’ learning if the person responsible for structuring the classroom pays close attention to the system maintenance and change aspects of the learning environment. Key questions to consider involve: How orderly and clear are the expectations for the activity? Who has ultimate control over learning and activity? and How responsive is the environment to change? p. 208

I recommend that teachers who plan to implement the classroom flip consider the following suggestions. First, the flip structure seems to be more productive when students have a choice between multiple ways of interacting with the content of the course outside of class. When the focus of the flip is on giving students the freedom to interact with the content according to their own learning style preferences, the flip seems to be more successful. p. 209

Second, if the flip is used in an introductory course, the in-class activities should be less open ended and more “step by step” in structure. If some activities are open ended, try to keep them brief: one to two class periods. Students in introductory courses will often have little tolerance for prolonged uncertainty in the course content and the course structure. In more advanced classes, students will be more willing to push through prolonged investigations, but the structure of the classroom must support their meaning making in the activity. p. 209

third recommendation. A flip classroom is structured so differently that students will become more aware of their own learning process than students in more traditional settings. Students will therefore need to have more space to reflect on their learning activities so they can make the necessary connections to course content. The teacher must structure a major component into the course structure that will allow for this reflection to take place and for the teacher to be able to see and comment on specific aspects of student reflection. This feedback cycle will be crucial for student learning. p. 209

It would be useful for the teacher in the flip p. 209

classroom to help students become self-aware of how they approach classroom activity and what their mind-set is toward that activity (using the aforementioned properties and dimensional ranges). This self-awareness could help students understand the difficulties they encounter, contribute to an increase in their self-efficacy for learning, and provide a way for the teacher to support students when they are struggling p. 210

in a flip classroom, we have mentioned it will be important for students to have a structure for reflecting on their different activities. Thus, at least weekly, it would be useful for the teacher to give writing assignments (such as 1⁄2-page reflection papers) where students address a very specific question designed by the teacher that ties together the major concepts of recent classroom activity. p. 210

This is the first area for future research that I suggest. What are the characteristics of course material that would lend itself to being taught in a course using the classroom flip structure? Are there certain characteristics of a group of students that would tend to make the classroom flip structure work better with them than with a group of students with different characteristics? These are just a few questions that could be pursued along these lines. p. 211

A final potential thread of investigation related to comfortability I want to suggest involves “pragmatic learners.” These learners are defined by Broad et al. (2004) as only being interested in completing assignments and not exploring implications of concepts. I believe it could be productive to investigate pragmatic learners’ comfortability in classroom learning activity. p. 212

REFERENCES p. 214