Download Effectiveness of Simulations & Screencasts in Enhancing Chemistry Learning and more Study notes Chemistry in PDF only on Docsity!
Title
Supporting Students’ Conceptual Understanding of Kinetics Using Screencasts and Simulations Outside of the Classroom
Authors
Ryan D. Sweeder,a^ Deborah G. Herrington,b^ Jessica R. VandenPlas b
a. (^) Lyman Briggs College, Michigan State University, 919 E. Shaw Ln, East Lansing, MI 48825, USA.
b.email: [email protected] Department of Chemistry, Grand Valley State University, 312 Padnos Hall, Allendale, MI 49401, USA
Abstract
Simulations have changed chemistry education by allowing students to visualize the motion and interaction of particles underlying important chemical processes. With kinetics, such visualizations can illustrate how particles interact to yield successful reactions and how changes in concentration and temperature impact the number and success of individual collisions. This study examined how a simulation exploring particle collisions, or screencast employing the same simulation, used as an out-of-class introduction helped develop students’ conceptual understanding of kinetics. Students either manipulated the simulation themselves using guided instructions or watched a screencast in which an expert used the same simulation to complete an assignment. An iterative design approach and analysis of pretest and follow up questions suggests that students in both groups at two different institutions were able to achieve a common base level of success. Instructors can then build upon this common experience when instructing students on collision theory and kinetics. Eye-tracking studies indicate that the simulation and screencast groups engage with the curricular materials in different ways, which combined with student self-report data suggests that the screencast and simulation provide different levels of cognitive demand. This increased time on task suggests that the screencast may hold student interest longer than the simulation alone.
Keywords
Kinetics, Simulations, Screencast, Eye-tracking, General chemistry, Assessment Development
Introduction
Simulation and animations use in introductory chemistry classrooms is becoming more commonplace. These electronic resources can make visible the otherwise unobservable particulate level interactions that are the underpinning of chemical phenomena. Therefore, these representations can help students explain important chemistry concepts (Akaygun and Jones, 2013). These results may reflect that the simulations provide a valid visualization of particulate level interactions; a level of chemistry that is a common challenge for students (Gabel et al. , 1987; Nurrenbern and Pickering, 1987; Sanger, 2005; Chittleborough and Treagust, 2007; Williamson, 2014). Thus, it is not surprising given that in-class use of both animations and simulations have been shown to improve students’ mental models (Williamson and Abraham, 1995; Sanger and Badger, 2001; Yezierski and Birk, 2006; Akaygun and Jones, 2013). Simulations can offer an advantage over animations in these contexts as the students can be actively involved in adjusting parameters and seeing how the system responds. However, animations offer a more controlled presentation that can, if well designed and used effectively in the classroom (Kelly and Jones, 2007; Tasker and Dalton, 2008), direct student attention to the important details of the molecular representation. As part of an ongoing research project, we have been exploring the relative effectiveness of students completing a guided assignment while using a simulation compared to viewing a screencast (an animation consisting of a recording of an expert narrated use of the simulation) as an outside of the classroom introduction to core chemistry concepts. In our previous work looking at the topic of solubility, we saw student learning gains from both the simulation use and the screencast, with the screencast showing slightly stronger learning gains (Herrington et al. , 2017). This study aims to understand how these two approaches to using the simulation can influence students’ learning of key concepts associated with kinetics.
Student learning of kinetics
Kinetics is one of the Anchoring Concepts in general chemistry. Fully understanding and manipulating chemical reactions requires a solid grasp of reaction rates, what factors affect them, and underlying theories (e.g. collision theory) (Cachapuz and Maskill, 1987; Justi, 2002; Cakmakci and Aydogdu, 2011; Talanquer, 2016). For these reasons, many studies have demonstrated that reaction kinetics is a challenging and important introductory chemistry concept for students (de Vos and Verdonk; Cachapuz and Maskill, 1987; Justi, 2002; Van Driel, 2002; Cakmakci, 2010). In their recent review, Bain and Towns (2016), conclude that most of the common misconceptions surrounding the learning of kinetics have been identified. Therefore, even though there have been several studies on pedagogical techniques to help students learn reaction kinetics (Justi and Gilbert, 1999; Justi, 2002), identifying better methods to teach this challenging concept is one of the leading problems. Given the ability of online simulations to present the molecular motion that is key in building deep understanding of kinetics, these hold great possibility for helping students learn kinetics without the confusion that static textbook depictions can introduce (Cakmakci, 2009).
Theoretical framework
Given that little is currently known about how screencasts or simulations can best be used to impact student learning of core chemistry concepts outside of the classroom environment, the goal of this study was to examine the following research questions:
- What are the impacts of outside-of-class usage of simulations or screencasts on students’ conceptual understanding collision theory and rates of reactions?
- How and where do students allocate attention while interacting with a simulation, as compared to a screencast, when coupled with a guided assignment?
Methods
Understanding how students use simulations in outside-of-class setting poses specific challenges. Principally, the instructional materials developed to support any learning activity are critical given that they must provide the guidance and support that an instructor’s probing questions afford in a classroom or laboratory situation. Identifying best practices for the development and use of these materials requires careful evaluation using both quantitative assessment of the resultant student learning as well as critical qualitative analysis of student responses to ensure that the prompts are effective in directing student attention while simultaneously avoiding potential misinterpretation. This demands an iterative approach to the development and refining of instructional and assessment materials. Further, for computer-based visualizations such as simulations, eye tracking can provide a complementary opportunity to deeply understand how student attention is allocated while engaging with the visual representation. Together these methods can provide deep insight into the student experience of learning while using a simulation.
Simulation
Kinetics instruction focuses on reaction rates and the variables that affect those rates. Accordingly, in the teaching and learning of kinetics students are frequently called upon to draw and interpret graphs involving concentration and time. However, a firm grasp of kinetics concepts requires students to be able to connect the graphs to what is happening at the particulate level. Thus, a simulation designed to help students develop this conceptual understanding of kinetics should include these multiple levels of representations and a means to support the connection between them. PhET Interactive Simulations are designed to provide multiple representations, including at the particulate level, connect to the real world, and allow student interaction and inquiry (Moore et al. , 2014). Therefore, the PhET Interactive Simulations Reaction & Rates simulation (Reactions & Rates) was selected to use for our investigation because it contains:
- A Single Collision tab, in which the user can shoot an atom into a molecule to initiate a reaction. The user can vary the speed and angle with which the particles collide and select reaction systems with different reaction coordinate diagrams (varying activation energy and Hrxn ). The user can also adjust the temperature up and down to see the change in particle motion and how that impacts the success of the collision.
- A Many Collisions tab, in which the user can mix differ quantities of particles from within each of the reactions systems in the first tab. In addition to the adjustable temperature and ability to start with a specific number of each species of particle, the tab provides assorted graphical representations of the current number of particles, including a strip chart.
- A Rate Experiments tab, which allows the user to precisely create a system with a specified number of starting particles. This tab was not employed in our investigation.
Assignment Design
Using the Backward Design (Wiggins et al. , 2005) approach outlined in Figure 1, the following key learning objectives related to understanding collision theory were identified:
- Describe collision theory and use it to explain changes in rates of reactions
- Explain how changing the temperature impacts the rate of a reaction
- Explain how changing concentration affects the rate of a reaction
a. What happens when the particles collide? b. Is there any reaction? (Y/N) What causes or prevents a reaction from taking place?
- Try adding more A (another 3 pumps) and watch for a minute or so. a. Is there any reaction now? Y/N b. What causes or prevents a reaction from taking place?
seem to have relatively flat lines here in the strip chart suggesting that no reaction has actually occurred. Well, let’s add in some more A and see if perhaps we can get a reaction to occur this way. So, we see that going up off the scale, so let’s adjust that so it fits. Well, and now it looks like we have a slight decrease in A and BC and an increase in the amount of C and AB that is present, and so now it appears that there has been some reaction that’s actually occurred.
Further revisions of the assignments, assessment questions and screencast were informed by analysis of student responses. This iterative assignment design and assessment cycle, as outlined in Figure 1, is critical to ensuring the quality and efficacy of the pre- assessment and follow-up questions. Analysis and revision help allow us to determine if the questions and prompts accurately evaluate student understanding, measure changes in understanding, direct student attention to the most salient features of the resource, and help them construct an understanding of the underlying concepts from a particle level perspective. Further, since simulations have limitations, student responses need to be critically examined to identify ways in which the limitations of the simulation may result in development of incorrect or inconsistent understanding. This is particularly important when students are manipulating the simulation themselves as these issues may not be immediately obvious to experts. For example, in the initial draft of the simulation assignment, students were prompted to increase the temperature and describe what happened to the total energy of the system. A surprisingly large number of students expressed that increasing the temperature resulted in no change in total energy, although many simultaneously expressed that the molecules moved faster. It was determined that this disconnect arose from the fact that in this simulation it was relatively easy for the students to max out the total energy line which meant it stopped increasing as the system was further heated, even though the molecules themselves continued to show the effects of the additional energy. Systematic analysis of student responses allowed us to identify this discrepancy between the answers of the screencast and simulations groups, diagnose its cause, and adjust the prompts in the simulation assignment to first have students decrease the temperature and answer a few questions. This then eliminated the previously identified confusing situation when they subsequently raised the temperature, as evidenced by the fact that we did not see this same answer on subsequent iterations of the assignment.
Designing questions to effectively measure learning is also a challenge. Closed response questions, such as multiple choice, can be appealing as they provide both question context and bounds for the answer and are easy to score. However, these questions can also be limit the detection of learning gains as multiple choice questions have been shown to overestimate student understanding (Lee et al. , 2011; Hubbard et al. , 2017). Our initial pre/post questions were predominantly multiple choice and resulted in very high scores (Figure 2). The inability of the students to accurately provide constructed responses on follow-up questions indicated that they were lacking the fundamental understanding that their closed response answers suggested. By replacing multiple choice questions with constructed responses, we have seen a decrease in student scores, but an improvement in the ability to discern students’ learning gains.
Figure 2 : Pretest and follow up scores across revisions
Classroom Study: Participants and Study Design
This study was reviewed and approved as exempt by our Institutional Review Boards (GVSU Ref. No. 16-012-H; MSU x15-799e). The participants for this study were students enrolled in General Chemistry 2 at one of two different large public institution in the Midwest region of the United States. Table 2 shows the number of participants, instructors and institutions involved in each iteration of the study. After each of the first two iterations, adjustments were made to the pretest, follow up questions, assignment, and screencast to address difficulties encountered by the students or to improve the sensitivity of the instruments to showing learning gains. Given iterative nature, the data analysis will focus predominantly on the 3rd^ iteration, except where noted.
and VandenPlas, 2014). These students were recruited from classrooms that had covered the relevant prerequisite chemistry content but did not participate in the larger classroom study. During the eye-tracking sessions, participants engaged in the same activities as students in the classroom study but did so in a condensed timeframe. During the eye-tracking session, students first took the pretest using paper-and-pencil and then completed either the screencast (n=13) or simulation (n=14) assignment, answering all questions embedded in the activity, while seated at a Tobii T60 eye tracking system. This system displays a stimulus on a 17-inch computer monitor and samples the participant’s eye position at a rate of 60 Hz. Participants sat approximately 24 inches from the monitor, and the system was calibrated to each participant prior to data collection. During the eye-tracking portion of the session, students were shown a split screen, with the simulation/screencast occupying the top half of the screen, and the corresponding assignment displayed on the bottom half (see Figure 3). Students were able to manipulate the mouse to control the simulation or pause/rewind the screencast as needed. They were also able to scroll through the assignment and follow-up questions and were asked to give all answers aloud to avoid the need for writing. Although the interviewer wrote down the student responses to the assignment questions as they were given, interviews were audio recorded to ensure that all student responses were captured accurately. Participants in both groups were given the opportunity to use the simulation for the follow-up questions if they wanted to do so. These sessions lasted approximately 30 minutes each.
Figure 3 Stimulus presentation for eye-tracking study, showing simulation at the top of screen and assignment at the bottom of the screen
The eye-tracking data were processed to identify fixations using the Tobii Fixation Filter (Tobii AB, 2016). Fixations were then mapped to two areas of interest (AOIs): one for the electronic resource on the top half of the screen and one for the assignment itself on the bottom half of the screen. For each of these AOIs, the total amount of time spent in fixations within the AOI (in seconds) and the total number of fixations within the AOI were calculated for all participants. Mixed-design ANOVAs were used to analyze the data. For these models, fixation time or number of fixations were used as dependent variables. Treatment (simulation or screencast) was used as a between-subjects variable, with AOI used as a within-subjects variable.
Results and Discussion
Learning gains
Overall, the learning gains between the pretest and follow-up questions for each of the two treatments were relatively small. A mixed-design ANOVA was used to compare student performance from pretest to follow-up based on treatment (simulation or screencast). This test shows a statistically significant main effect for time (F1,252 =11.0, p<0.001, η p^2 = 0.04) indicating that, overall, students improved from an average of 2.59 to 2.87 on a 5-point scale. However, neither the main effect for treatment or interaction effect of treatment and time were found to be significant. This suggests that both screencast and simulation treatments were equally effective in improving student performance. Examining the pretest and follow-up scores in Figure 4 separated by both treatment and location, it is clear that although both of the classes completing the screencast showed improvement, the gains were predominantly achieved by the class that had a lower pretest score. This result is actually quite fascinating, as it suggests that the impact of this particular intervention is to help bring the class average to a relatively consistent level regardless of the pretest scores.
Figure 4 : Scores on pretest and follow-up questions
Learning objective 2 – impact of temperature change
- Which of the following is the primary reason for a higher reaction rate at higher temperatures? a. The higher temperature lowers the activation energy, so more particles can pass over the potential energy barrier. b. Particles increase in size with increasing temperature, providing more surface area for potential collisions. c. There is a larger fraction of molecules with sufficient kinetic energy to react. d. The likelihood that molecules and ions will have the required orientation increases at higher temperatures. 2. The yeast in bread dough causes a reaction that converts sugar into carbon dioxide. Why does the dough rise faster in a warmer area?
Learning objective 3 – effects of changing concentration
- The reactants for the reaction X + Y 2 XY 2 are shown below. In the blank box provided, draw a picture of a set of reactants (X + Y 2 ) that would result in a faster reaction rate. 3. Why does a glowing splint of wood burn only slowly in air, but rapidly burst into flames when placed in pure oxygen?
- Most chemical reactions do not have a constant rate. Most reactions tend to slow down as the reaction progresses from reactant to product. Suggest one reason that the reaction may get slower as the reactants get converted to product, and explain how this slows down the reaction rate (assume the reaction occurs only in the forward direction). 4. Most chemical reactions do not have a constant rate. Most reactions tend to slow down as the reaction progresses from reactant to product. Suggest one reason that the reaction may get slower as the reactants get converted to product, and explain how this slows down the
reaction rate (assume the reaction occurs only in the forward direction).
Given the gains on question 1 and knowing that there was little to no difference in pre- and follow-up, there must be a decrease in score on other paired questions. Both treatments had lower follow-up scores on Question 2. This result is perhaps not surprising given the change in question format from multiple choice on the pretest to constructed response on the follow-up. On the pretest, the students were simply required to identify the correct choice regarding how increasing the temperature increase the reaction rate, with 41% choosing the correct answer. However, on the follow-up question, the students had to construct their own answer and a full score was only given if students cited both an increase in the number of collisions (due to moving faster) and also that there would be more energy in the collision. Only 12% of the students identified both components, with another 32% of the students citing just one of the two factors and receiving half credit. The most common incorrect answers cited an increase in energy without linking it to the reaction (38%) or indicating that when temperature increases volume increases (9%). Overall, this led to a statistically lower score on the follow up question (0.28 vs. 0.41, F1,262 =11.6, p<0.001, η p^2 =0.04). As research has demonstrated that students are often able to answer multiple choice questions on a given topic correctly with an incorrect or partially correct conceptual understanding using test taking strategies or other heuristics where they are unable to correctly answer a constructed response question on the same concept (Funk and Dickson, 2011; Hubbard et al. , 2017; Couch et al. , 2018), the pretest score could certainly be viewed as an over estimate of the student understanding. Taken together, these results suggest that though students may be making some progress on learning objective 2, many still lack a full conceptual understanding of this idea.
Eye-tracking Study
Research question 2 focuses on how students allocate attention while interacting with a simulation or screencast and working through a guided assignment. In this study, fixation time was used as a measure of attention allocation. Previous research has shown that the more time an individual spends fixating on an object, the more attention they have paid to this object (Goldberg and Kotval, 1999; Holmqvist et al. , 2011). The number of fixations made by an individual also can be used as a measure of attentional focus. In general, a high number of fixations has been shown to correlate both with a low level of expertise and a low level of search efficiency (Goldberg and Kotval, 1999). A high number of fixations may also indicate an unfocused individual, who is spreading their attention over a number of objects indiscriminately. In this study, the number of fixations and total fixation time were found to be significantly correlated for both the resource and the assignment (r=0.82, p<0.001). This is consistent with the results of other eye-tracking studies on student problem solving in chemistry education (Stieff, 2011; Tang and Abraham, 2016). For this reason, the remainder of the analysis focuses on fixation duration alone. Total fixation durations for each AOI are given in Table 4.
<.001 two-tailed). The students from both groups then report spending an equivalent amount of time on the follow-up questions (15. vs. 13.8 min; Mann-Whitney U = 17757, p = .599 two-tailed). The seeming disconnect between these two results may be an indication that the simulation demands a higher intensity of attention and greater mental effort, leading students to overestimate the time that the activity required due to its perceived difficulty (Tzetzis et al. , 2001). Therefore, it may be that the greater mental effort required to answer the questions through manipulation of the simulation makes it feel to the students as though the time on task is longer than it really is. It is also possible that the way students engage with the assignments at home and in a laboratory setting are different. For example, students at home may view the screencast in a single setting, whereas those engaged in exploration via the simulation alone may take more breaks while working on the assignment (an option not exercised by students in the laboratory setting), which may lead to inflated reports of time on task.
Other interesting patterns arose out of the analysis of the time data. There was weak, but significant negative correlation between a student’s pretest score and the amount of time they reported spending on the simulation assignment (-.204, p = .049) suggesting that students who were least successful on the pretest spent more time working on the assignment. This would be meaningful from an instructional standpoint, suggesting that students are innately recognizing their level of success and adjusting their effort accordingly. Separately, there was a strong correlation between the amount of time spent on the assignment and the posttest follow-up questions for the simulation students (.721, p <.001). However, this relationship was not present for those completing the screencast (.163, p =.074); it seems that the length of the screencast drives the amount of time spent on assignment, suggesting that there is very little additional exploration that occurs.
Figure 5: self-reported working times in minutes.
Conclusions and Implications
The use of simulations and animations in the teaching and learning of chemistry has notably impacted students’ abilities to visualize the motion and interactions of particles in different situations and how those motions and interactions changes with the manipulation of different system variables (Lowe, 2003). As Johnstone pointed out in his seminal paper (Johnstone, 1982), the ability to explain what is happening at the macroscopic level based on particle motion and interactions is crucial for the conceptual understanding of chemistry that separates experts in the field from novice learners. Though, particularly in the case of the PhET simulations, much research has gone into their design to provide an environment that encourages and scaffolds student exploration and development of conceptual understanding (Kelly et al. , 2013), it still remains that novice learners may fail to notice some of the most germane aspects of the particle motions and interactions without proper scaffolding and direction (Jones et al. , 2005). This research indicates that it is possible to design assignments with appropriate scaffolding prompts to help students develop, on their own and outside of class, some foundational understanding of collision theory and how variables such as concentration and temperature affect the rate of collisions and hence reactions. Yet an iterative process is needed to develop the quality materials that provide enough scaffolding to focus student on the most salient aspects of the simulations and help them construct a particle level understanding of the underlying concepts without over scaffolding the interactions such that students are not actively engaged in their learning. The most important findings is that the completion of this assignment as an introduction to collision theory/kinetics appears to bring classes on average to a common starting point (see Figure 4).
This finding has important implications in the teaching for the majority of us that find ourselves instructing classes of students with vastly different background situations and preparation in chemistry. Though it is dependent upon the student actually engaging meaningfully with the activity, our data from multiple classes with different instructors and different institutions suggests that having students engage with the simulation or screencast version of this assignment prior to instruction has the potential to bring students in the class to approximately the same level of understanding. This allows instructors to begin their instruction of collision theory and kinetics with students’ having a common experience on which to build. This is something that is not often the case, particularly in large lectures with a diverse student population.
Results from the eye-tracking study suggest that at least part of the students’ success in both the screencast and simulation treatment may come from their engagement with the electronic resource itself. Students in both groups spend significantly more time focusing on the electronic resource than they do on the assignment questions, indicating that they are using the resource to answer questions
The authors would like to thank the instructors and the students who made the collection of data possible. We would also like to thank Marissa Biesbrock and Kristina Pacelli for collecting the eye-tracking data. This material is based upon work supported by the National Science Foundation under Grant Nos. 1705365 and 1702592.
Appendices
Appendix 1: Simulation Assignment
Name: _________________________
Reaction Rates Guide
- Go to the PhET simulation Reactions and Rates (http://phet.colorado.edu/en/simulation/reactions-and-rates) - and choose Run Now. Note: You will need Java installed on your computer for this. Java does not work with Chrome so you will need to use another browser. After you install Java you will need to add the above website to the java exception site list which can be found under the Security tab of the Java Control Panel. Instructions for how to find the Java control panel and add this site to the exception list for both Macs and PCs can be found in the Java Instructions document.
- What is the clock time as you start the assignment? ______________________ Part A: Single Collision
- Make sure you are on the Single Collision tab at the top.
- Select the second reaction on the far right (with A being green and BC containing a red and blue atom).
- Click on the “+”sign in the top right hand corner of the Separation View and Energy View boxes so that you can see the separation of the particles and the reaction energy diagram in the boxes.
- Pull down the plunger about ½ way. What happens to the total energy line (green line) on the reaction energy diagram?
a. Release the plunger. **Does a reaction take place?** Yes/No **What causes or prevents a reaction from taking place?** __________________________________
- Hit the “ Reload Launcher ” button at the far right and then pull the plunger down all the way and release.
a. What happened to the total energy line when you pulled the plunger down all the way?
b. When you released the plunger did a reaction take place? Y/N What causes or prevents a reaction from taking place? __________________________________ c. Watch the particles collide for a minute or so, do they react every time they collide? Y/N d. What factors are different between when a reaction happens and when one doesn’t?
- Lower the temperature.
a. What happens to the particles? __________________ b. What happens so the total energy of the system? ________________________ c. What happens to the particles and the total energy of the system when the temperature is raised?
