Download Analysis of Capsim Simulations in MBA Program: Business Simulations in Higher Education and more Exams Business in PDF only on Docsity!
The Business Simulation Paradigm: Tracking
Effectiveness in MBA Programs
Bryan Forsyth
Colorado Technical University
Christina Anastasia
Colorado Technical University
This article illustrates how a program like the Capsim Business Simulation can impact any business program in the Capstone phase as a test of effectiveness of the school’s program. The simulation determines the level of the skill sets of students in terms of six functional domains including Accounting, Finance, Human Resources, Marketing, Operations and Strategy at program conclusion. This longitudinal study done over three years demonstrates a number of significant findings and implications for practice with the most important seeming to be that it is best to allow eleven weeks or more in a course for the most desired outcomes.
INTRODUCTION
Today, higher learning institutions are seeking ways to measure learning outcomes in Master’s programs. The Higher Education Opportunity Act of 2008, requires Universities to meet the strict demands through accreditation meaning that they must be able to provide an education that will benefit the student to find meaningful employment as a result of his or her degree (Neely & Tucker, 2012). In the 21 st^ century, technology allows schools of higher learning to incorporate virtual business simulations in to the learning platform. In an effort to create a learning environment where students can successfully apply academic lessons to real world organizational environments, Universities are turning to concepts such as business simulations. Ahn (2008) points out that approximately 95% of The Association to Advance College Schools of Business (AACSB) schools are using some type of business simulation to track learning outcomes (p. 17). Virtual business simulations are relatively new to MBA programs and many Universities are embracing the technology to help track learning outcomes and better prepare Master’s degree students for life after school. Previous research on business simulations has had mixed results. For example, Frezzo, Behrens, and Mislevy (2010) suggest that there are problems with the social interaction or “dispersed social systems” that may cause teams to fail. Neely & Tucker (2012) found that simulations are too complicated to work effectively. Graziano (2003) researched the concept of virtual business simulations to understand the learning outcomes and found that these types of business simulations can be an effective tool for engaging students in preparation for real work scenarios. Essentially, the findings support the idea that virtual business simulations add to the experiential elements of higher learning. The mixture of
different research findings presents one common agreement which is the need for continued research on the effectiveness of business simulations as a requirement for a degree.
Background of Problem In 2013, the Capsim business simulation was introduced to a major University as a means for measuring learning outcomes for Master’s degree students. The simulation was used in five consecutive sessions in the final Capstone course. Professors were given two brief online training programs to prepare for the classes. Students were made aware of the new process as they entered their final course for their degree program. While previous courses in the programs allowed students to work individually on 90% of the course content, the Capsim simulation required students to be grouped into teams from the first day of class. The teams would be introduced to the Capsim simulation during the first week of class and given a practice period of one week to familiarize themselves with the simulation. In the second week, teams were required to begin making decisions in the simulation in competition mode. At this point, teams were expected to have selected a strategy, assign decision making roles, and start entering decisions in the competition rounds of the simulation.
Problem The professors structured their classes to familiarize the students with the simulation as early as possible in the class. Teams were assigned prior to the start of the class in order to give students the opportunity to contact team members the moment they entered into the classroom. Professors also provided students with simulation guides and scheduled lectures at the start of the class covering the basics of the simulation. While students were not required to attend the lectures, the attendance was high. As the teams preceded into the competition rounds, the Professors noted several problems with the teams including interaction, in-fighting, and lack of role clarity. In the subsequent sessions, the instructors worked to create cohesive groups from day one in the simulation. While every effort was made to work with the teams both inside and outside of the classroom, many of the issues remained prominent throughout the simulations. Prior to the end of the team competition rounds, students were then required to complete the simulation comprehensive exam. A recurring theme showed in the exam, which was an overall low score in each of the classes compared to the national averages. This is something no University wants to see in their numbers and is an indicator of much deeper problems in the programs leading up to the Capsim Capstone. The general problem is the lack of sufficient time to familiarize students with the Capsim simulation in a manner that would prepare them for the final exam. The main problem is a consistency in the lower than average results on the CompXM final exam. The Professors have experimented with different ways to address this problem and have come up with research questions as well as hypotheses to test which is what follows.
Purpose/Proposition The consistency in the low CompXM scores compared to the industry average could be related to the degree program at the University, meaning that the scores may depend on the type of Master’s degree the student is earning. The purpose of this paper is to correlate the University program objectives to the Capsim simulation objectives by reviewing the simulation outcomes of the five consecutive sessions using Capsim and offer recommendations for future use of the simulation. Also, the experiments conducted thus far to try and increase overall scores will be featured at the end of this paper.
Research Questions and Hypotheses
R1: Are specific Master’s degree programs designed to align with the business simulation?
teams in the industry. The report provides a financial summary including cash flow, balance sheet, and income statements. The next section includes a production analysis for the industry segments. If the HR and TQM modules have been purchased, this information is provided in the next section followed by an ethics report. The remainder of the report is the annual report for the individual team. Capsim offers two levels of simulation including Capstone and Foundation. The traditional Capstone simulation is directed towards players that have strong financial backgrounds and are working on MBAs. The Foundation simulation is directed towards undergraduates. However, Foundation can also be used for Master’s Degree students that have majors focused on the softer sciences such as Human Resources, Organizational Change, and Marketing. The differences are presented in the chart below.
Capstone Foundation $100 million company, with five average products, in very different market segments,
40 million company that has no clear direction and poor financial results. Traditional, Low End, High End, Performance low tech and high tech research and development, production, marketing and finance.
Research and Development, Marketing, Finance and Production. Additional modules – in human resources, quality management and sustainability, labor negotiations and advanced marketing
Human Resources and Total Quality Management can be added.
With five to eight years to build success, With five to eight years to build success, (CAPSIM Professor Guide, 2013, p.3)
This paper will discuss the Capsim Foundation simulation currently being used to assess learning in a major University.
Description of Degree Programs Using Capsim The goal of this paper is to correlate the University program objectives to the Capsim simulation objectives. To do this, an overview of the University Master’s programs is discussed. The programs currently using Capsim for the Capstone course include MBA with an emphasis on Finance, MBA with an emphasis on Human Resources, MBA with an emphasis on Marketing, MBA with an emphasis on Technology Management, MBA with an emphasis on Operations/Supply Chain Management, and a general MBA. In addition, there are some non-MBA programs that have been involved such as a Masters in Management as well as Organizational Leadership and Change. For each of these Masters levels programs the overall objectives are the same. The programs should help the student learn how to manage across various business disciplines, apply business theory, concepts, and methods for current or future positions. The final objective is to enhance the mission and values of an organization. The overall program competencies are similar for all MBAs and Master’s degree programs. Synthesize ethics, leadership, strategy, critical thinking and reasoning skills in a business environment. Integrate professional applied research in order to validate and justify decision making. Formulate information in order to communicate effectively across the appropriate channels. Utilize appropriate technology to maximize efficiency and effectiveness in order to accomplish organizational goals and objectives. Combine the major functional areas of business administration including the critical skills necessary to solve business problems, individually and collaboratively. There are some differences in each of the programs. The chart below reflects these programs in question:
Program MBA Non-MBA Non-MBA Concentration Business Administration
Organizational Leadership & Change
Management
(Anastasia, 2013)
Students from any one of the above listed programs were required to compete using the Capsim simulation for their final Master’s program Capstone course. The idea was to have a measurement in place that would provide an analysis of the overall learning for the Master’s program. Prior to beginning the new process, two instructors were selected to take the Capsim faculty training. The training consisted of several live sessions with the Capsim trainers to familiarize the instructors with the simulation itself and how to analyze the decision results and reports. Also available to the instructors were a series of videos and an instructor guide. Upon the completion of the training, the University began the process of enrolling Master’s program students in the final Capstone class which was updated to incorporate the Capsim simulation. The following section will analyze the results of the simulation for a period of five consecutive Capstone sessions using Capsim Foundation simulation.
ANALYSIS OF CAPSIM RESULTS
Description of the Scoring As stated earlier, the Foundation simulation is graded using a balanced scorecard. The rounds are broken down by sections including market share, margin, profits, emergency loans, working capital, forecasting, customer satisfaction, financial structure, wealth creation, and productivity. There are a possible 1000 points that can be earned by a team in a round.
Points Summary Points Value Market Share 100. Margins 100. Profits 100. Emergency Loans 100. Working Capital 100. Forecasting 100. Customer Satisfaction 100. Financial Structure 100. Wealth Creation 100. Productivity 100. Total Points 1000. (Capsim Simulation Reports, 2013)
The points listed above are then broken down into a balanced score card which converts these results into the final score. For this analysis Capstone students were required to complete four of the eight rounds.
When the results were sorted by Master’s program in this round, there was little differentiation in the average scores.
(Capsim Simulation Reports, 2013)
Session Two 2013 In the second session using the Capsim Foundation simulation, 52 students completed the course. The median score for all of the Master’s Degree students on the CompXM was 380 out of 1000 points.
(Capsim Simulation Reports, 2013)
When the results were sorted by Master’s program in this round, there was a noticeable differentiation between students in the Executive MBA program compared to other Master’s programs.
(Capsim Simulation Reports, 2013)
Session Third 2013 In the third session using the Capsim Foundation simulation, 62 students completed the course. The median score for all of the Master’s Degree students on the CompXM was 349 out of 1000 points.
(Capsim Simulation Reports, 2013)
When the results were sorted by Master’s program in this round, there was a noticeable differentiation between students in the Executive MBA program compared to other Master’s programs.
(Capsim Simulation Reports, 2013)
Session Five 2013 In the fifth session using the Capsim Foundation simulation, 47 students completed the course. The median score for all of the Master’s Degree students on the CompXM was 412 out of 1000 points. The average for the students in question was 41% compared to the national average of 60% to include the functional domains of Accounting, Finance, Human Resources, Marketing, Operations and Strategy which is an improvement over earlier scores in CompXM against the national average.
(Capsim Simulation Reports, 2013)
When the results were sorted by Master’s program in this round, the Executive MBA students were outranked by the other Master’s program students.
(Capsim Simulation Reports, 2013)
The Early 2013 Results Compared The comparison of scores over the five consecutive sessions shows a slight improvement.
(Capsim Simulation Reports, 2013)
Moving Longitudinally to 2016 Three years later using the Capsim Capstone simulation, 32 students completed the course in the latest session. The change was made to include only MBA students in the course so the dynamic did change and some improvement can be attributed to that. There were no other changes to consider with how the course was administered to students from 2013. The median score for all of the MBA Degree students on the CompXM was 427 out of 1000 points. The average for the students in question was 44%
Null Hypothesis (H0): The type of program the student is enrolled in does not have any impact on where they place in the standings after they take the CompXM final exam.
Hypothesis (H1): General (non-Finance) MBA program participants will do better in the CAPSIM simulation than other students who are in a purely Management program. Hypothesis (H2): Finance MBA program participants will do better in the CAPSIM simulation than all of the rest of the students.
Initially, the Professors involved in this activity had a suspicion that the degree program that the student had been through to date would be the big differentiator related to success in the CAPSIM simulation. The thought was that anyone in an MBA program would do better than one who was simply in a Management program; that someone in a Finance emphasis would do better than all the others. While it does appear that the latter is more so true than the former, when comparing MBA to non-MBA students, final scores on the CompXM exam were ranked and some of the lowest scores were MBA students as well as the highest. The students in the Management program fell in between in every case examined for the five classes in question. So, in essence, there was a failure to reject the null hypothesis (H0) with the first alternative (H1) hypothesis which stated that General (non-Finance) MBA program participants will do better in the Capsim simulation than other students who are in a purely Management program. However, in taking a closer look at the Finance majors, they were all toward the top end of the scale for standings overall. This might lead one to believe that there is an acceptance of the second alternative hypothesis (H2) as it relates to Finance majors. Another variable that could impact these findings is the background of the student as well. Some of the students that were enrolled in non MBA programs did better than average on the CompXM than some of the MBA students and it was discovered that they had owned their own businesses previously where they had to manage all of the elements as they would in CAPSIM. In this case the question “Is experience the best teacher?” comes into the equation one would think. With the student population being studied here there is a well noted lower than average score as compared to the national norms. It is not known what else is done by other Professors in other schools to prep their students for CompXM. Could it be that they “teach the test”? If that is the case, then the lower than average numbers we find here might just be indicative of such a thing. In this case, the professors did not “teach the test” but allowed for the students in teams to work together and figure it out. In the last few classes, one Professor set the simulation up for three practice rounds and then four competition rounds along with an option to do the footrace as mentioned above. How about the old saying “practice makes perfect”? It is this writer’s belief that the more people learn by doing with guidance, feedback and redirection along the way, the more they are capable of doing it on their own when the time comes. Another old saying comes to mind about teaching a man to fish. The class is set up for five and a half weeks which has not been done before using this simulation. Given that this is the case and the student population in question here as noted above being lower than the national average in the competencies shown for Business Acumen in CAPSIM, it has been a grueling trial for Professors and students involved. The qualitative data below illustrates that the students struggle a lot when getting started but come into their own and in most cases, really enjoy the simulation and what it has done for them in terms of closing out their Master’s Degree programs.
Student Feedback Students were asked to provide their thoughts on the Capsim simulation up completing the CompXM. One student stated (non MBA student):
“I think this is a really cool concept. It would have been nice to be able to learn with this tool throughout the course of the 18 months that I was in school. It would also make a better transition because for the last 17 1/2 months I have had pretty much the same
homework schedule of due dates, meeting times and time allotment in my busy schedule. It has rocked my world because it has changed. I have spent well over 20 hours a week comprehending and sulking.”
“It was a little over whelming to learn a program, work with a group from the get go and not be in control of decisions that were being made. Because we were making decisions individually it was putting a lot of pressure on the COO. I can say that tonight our group finally pulled together. It is still over whelming but being able to talk live and have access to one screen so that we all can help make decisions helped all of us learn tremendously. I actually felt part of the group and was listened to.”
“The book helped but I can see parts of each of my classes in this SIMs class. I think that I would be able to see better connections and grasp the power of the program full circle if I was exposed to it prior or during my tenure at….”
“Everything connects... If R&D doesn't make innovative products profits won't increase. Marketing doesn't have the correct stats and market predictions, production could over or under produce. If that happens stocks go wacky and you can be knee deep in debt or better yet swimming in money.”
“ The value of this experience is a better business sense to make better executive decisions in many aspects. This creates a real life scenario with the fringe benefit of hopefully not getting fired (failed).”
An MBA student who completed two rounds of the Footrace and is also a Finance major wrote:
“The footrace is very helpful to understand CAPSIM. In the footrace, it was hard to remember that I was now Andrews and not Ferris. At about round 4 I decided to make this Excel sheet and backfilled it in afterwards. I wanted to track and project my R&D products and using excel allows for you to put in some formulas. I projected the customers’ requirements and demand down at the bottom to help with sales forecasting. I did not look for the TQM in round 1 so I missed the start of that. I noticed after a few rounds that I could taper off the TQM funds. For HR I started to max out both training and recruiting at round 2 and left it alone to help with cost reductions.”
“ The footrace helped me to understand CAPSIM and I had to reread the guidebook several times. If you can master the inventory you will do great. I also checked the balance scorecard before I processed each round to ensure that I was getting highest score possible. This actually stopped me from adding automation and reducing all of my debt because it dropped the score. It was interesting to see that my potential market sales dropped each year I changed the product. My success is truly from reading the guidebook and using the scorecard. I hope that my Excel sheet helps to understand CAPSIM.”
Egereonu (2011) conducted a case study of the student perceptions of business simulations and found that there is added stress to students in a simulated business environment due to lack of time for preparation and implementation in the typical college session. The comments support this conclusion and reflect in general how most students felt about their experience and demonstrate that a lot of learning occurred. Some students did not like Capsim at all and felt it was not applicable to them personally but they still learned and did well in the CompXM final exam in a number of cases. What does all of this mean?
increase rigor and lasting results that will properly prepare the student for this final test of their skills when they leave the University. Something to be avoided at all costs is to have this simulation that is designed to be done in eight or more weeks done in little more than five weeks. After the three years of time this study has been ongoing, there have been no significant changes in CompXM score improvement against the national norms with the five and one half week term format. However, internally, there was evidence that using the Footraces early on in the course, contributes to better scores overall because there seems to be a strong correlation between more practice and better results. Always allow for two practice rounds and eight competition rounds minimum which would typically take an eleven-week class or more to accomplish successfully. Always take a close look at the courses in the MBA program to determine whether what is being taught and reinforced in those courses reflects what the students will see in the simulation regarding decision- making and answering Board Query questions. If these findings are not considered, you will not be likely to have good success with a Business Simulation such as Capsim and if you do not have success with this simulation it is a strong indicator there are problems with the effectiveness of your Business (mostly MBA type of) programs.
REFERENCES
Ahn, J. H. (2008). Application of experiential learning cycle in learning with a business simulation game (3314573). Capsim Foundation Team Member Guide. (n.d.). Retrieved 2013, from http://www.Capsim.com Capsim Professor Guide. (n.d.). Retrieved 2013, from http://www.Capsim.com Capsim Simulation Reports , (n.d.). Retrieved 2013/2016, from http://www.Capsim.com) Egereonu, F. O. (2011). Business simulation: An exploration of the virtual enterprise program. UMI Dissertations Publishing. doi: Frezzo, D. C., Behrens, J. T., & Mislevy, R. J. (2010). Design Patterns for Learning and Assessment: Facilitating the Introduction of a Complex Simulation-Based Learning Environment into a Community of Instructors. Journal of Science Education and Technology , 19 (2), 105-114. Meyer, A. (2008, February). Failure can boost confidence: Capsim Management Simulations offers business students mock problems to prepare them for real challenges. Chicago Tribune [Chicago], p. 1. Neely, P., & Tucker, J. (2012). Case Study. An examination of the decision making process for selecting simulations for an online MBA program. Emerald Group Publishing , 55 (2), 128-138.
