Download Lack of Progress in Programming Languages & Debugging Tools Based on ESP & HCI Research and more Lecture notes Linear Programming in PDF only on Docsity!
It is somewhat surprising that in spite of over 30 years of research in the areas of empirical studies of programmers (ESP) and human-computer interac- tion (HCI), the designs of new programming languages and debugging tools have generally not taken advantage of what has been discovered. For example, the C#, JavaScript, and Java lan- guages use the same mechanisms
for looping, conditionals, and assignments shown to cause many errors for both beginning and expert programmers in the C language. Systems such as MacroMedia’s Director and Flash, Microsoft’s Visual Basic, and general-purpose program- ming environments like MetroWerks’ CodeWarrior and Microsoft’s Visual C++, all pro- vide the same debugging tech-
niques available for 60 years: breakpoints, print statements, and showing the values of vari- ables. Our thorough investigation of the ESP and HCI literature revealed many results that can be used to guide the design of new programming systems, many of which have not be utilized in previous designs. However, there are many significant gaps in our
COMMUNICATIONS OF THE ACM September 2004/Vol. 47, No. 9 47
An environment that works the way nonprogrammers
expect is more inviting and helps users become more
confident and productive.
Over the last six years, we have been working to create programming
languages and environments that are more natural, or closer to the way people
think about their tasks. Our goal is to make it possible for people to express
their ideas in the same way they think about them. To achieve this, we have
performed various studies about how people think about programming tasks,
both when trying to create a new program and when trying to find and fix
bugs in existing programs. We then use this knowledge to develop new tools
for programming and debugging. Our user studies have shown the resulting
systems provide significant benefits to users.
NATURAL PROGRAMMING
LANGUAGES AND
ENVIRONMENTS
By Brad A. Myers,
John F. Pane and Andy Ko
48 September 2004/Vol. 47, No. 9 COMMUNICATIONS OF THE ACM
knowledge about how people reason about programs and programming. For example, there has been very little study about which fundamental paradigms of computing are the most natural or what questions people ask when debugging. We are performing user studies which investigate these questions. It is in this context that we developed the Natural Programming design process, that treats usability as a first-class objective by following these steps:
- Identify the target audience and the domain, that is, the group of people who will be using the sys- tem and the kinds of problems they will be work- ing on.
- Understand the target audience, by studying the actual language, techniques, and thinking they naturally use when trying to solve problems. This includes an awareness of general HCI principles as well as prior work in the psychology of pro- gramming and empirical studies. When issues or questions arise that are not answered by the prior work, conduct new user studies to examine them.
- Design the new system based on this informa- tion.
- Evaluate the system to measure its success, and to understand any new problems the users have. Redesign the system based on this evaluation, and then reevaluate it, following the standard HCI principle of iterative design.
Thus, the Natural Programming approach is an application of the standard user-centered design process to the specific domain of programming lan- guages and environments. This article provides an overview of some of the work of the Natural Programming project.^1 We explore why naturalness might be better for develop- ers and what might be more natural in programs for graphics and data processing based on initial user studies. The results were used in the design of a new language and environment called Human-centered Advances for the Novice Development of Software (HANDS). We also discuss our survey of program- mers creating small and medium-size programs using a different environment called Alice (see www .alice.org).
Why Natural Might Be Better The premise of our research project is that program- mers will have an easier job if their programming tasks are made more natural. By “natural,” we mean “faithfully representing nature or life,” which here
implies it works in the way people expect. By “nat- ural programming” we are aiming for the language and environment to work the way that nonpro- grammers expect. Why would this make programming easier? One way to define programming is the process of trans- forming a mental plan in familiar terms into one compatible with the computer [3]. The closer the lan- guage is to the programmer’s original plan, the easier this refinement process will be. This is closely related to the concept of directness that, as part of direct manipulation , is a key principle in making user inter- faces (UI) easier to use. UI designers and researchers have been promoting directness at least since Ben Shneiderman identified the concept in 1983, but it has not even been a consideration in most program- ming language designs. Conventional programming languages require the programmer to make tremendous transformations from the intended tasks to the code design. For exam- ple, a typical program to add a set of numbers in C uses three kinds of parentheses and three kinds of assignment operators in five lines of code, whereas a single “SUM” operator is sufficient in a spreadsheet [2]. We argue that if the computer language were to enable people to express algorithms and data more like their natural expressions, the transformation effort would be reduced. Similarly, debugging activities could benefit from being more natural. Research describes debugging as an exploratory activity aimed at investigating a pro- gram’s behavior, involving several distinct and inter- leaving activities [12]:
- Hypothesizing what runtime actions caused failure;
- Observing data about a program’s runtime state;
- Restructuring data into different representations;
- Exploring restructured runtime data;
- Diagnosing what code caused faulty runtime actions; and
- Repairing erroneous code to prevent such actions.
Current debugging tools support some of these activities, while hindering others. For example, break- points and code-stepping support observation of con- trol flow but hinder exploration and restructuring, whereas visualization tools help restructure data but hinder diagnosis and observation [5]. Yet none of these tools support hypothesizing activities. The argu- ment behind our Natural Programming approach to debugging is that support for such question-related activities will significantly improve success. If pro- grammers have a weak hypothesis about the cause of (^1) More details are available from other papers at www.cs.cmu.edu/~NatProg). a failure, any implicit assumptions about what did or
to use. To avoid biasing the subjects’ answers, the study materials were constructed with great care, using graphical depictions and terse descriptions of problem scenarios. One study used the PacMan video game and another used database access scenar- ios more typical of business programming tasks. Some observations from these studies were:
- An event-based or rule-based structure was often used, where actions were taken in response to events. For example, “When PacMan loses all his lives, it’s game over.”
- Aggregate operators (acting on a set of objects all at once) were used much more often than iterating through the set and act- ing on the objects individually. For example, “Move everyone below the 5th place down by one.”
- Participants rarely used Boolean expressions, but when they did they were likely to make errors. That is, their expressions were not correct if interpreted according to the rules of Boolean logic in most programming lan- guages.
- Participants often drew pictures to sketch out the layout of the program, but resorted to text to describe actions and behaviors.
Additional details about these studies are reported in [9].
The HANDS Environment and Language The next step was to design and implement HANDS, a new programming language and envi- ronment. The various components of this system were designed in response to the observations in our studies as well as prior work. HANDS uses an event-based language that fea- tures a new model for computation, provides queries and aggregate operators that match the way nonpro- grammers express problem solutions, has high visibil- ity of program data, and includes domain-specific features for the creation of interactive animations and simulations. In HANDS, the computation is represented as an agent named Handy, sitting at a table manipulating a set of cards (see Figure 1). All the data in the system is stored on these cards, which are global, persistent, and visible on the table. HANDS is event-based—a programming style that most closely matches the problem solutions in our studies. It has full support for aggregate opera- tions—all operators can accept lists or singletons as
operands. Lists can be generated as needed, by using query operators that search all of the cards for the ones matching the programmer’s criteria. Queries and aggregate operations work in tandem to enable the programmer to concisely express actions that would require iteration in most languages. For example, the following is a typical HANDS statement combining queries and aggregates:
set the nectar of all flowers to 0
To examine the effective- ness of HANDS, we con- ducted a user study. Ten-year-olds were able to learn the HANDS system during a three-hour ses- sion, and then use it to solve programming problems. Children using the full-featured version of HANDS performed significantly better than their peers who used a version modified to be more like typical pro- gramming systems. Additional details about this study are reported in [9].
Debugging Studies Our studies of the language and structure that peo- ple use helped us to design a more natural program- ming language. We are also performing studies of how programming environments can help program- mers avoid making errors, and help them find and fix the errors they have already made. An important part of these studies is determining what causes program- mers to make errors in the first place. We have integrated many strands of prior research on human error, and have found that errors are ulti- mately caused by long chains of breakdowns that hap- pen for one of two reasons: some breakdowns occur in the programmer’s head, such as using an inappropriate
50 September 2004/Vol. 47, No. 9 COMMUNICATIONS OF THE ACM
Figure 2. The Alice programming environment with a “why didn’t” question in progress.
strategy, or having a misunderstanding about a partic- ular aspect of a programming language. The other type of breakdown is caused by things outside of the pro- grammers’ head, in the programming language and environment. For example, when it is difficult to inspect the values of variables at runtime, program- mers may have breakdowns in debugging. Or, when a language supports different meanings for the same text (such as the “+” operator in Java), programmers may accidentally introduce errors.
Thus, to prevent errors, program- ming environments should help pre- vent these types of breakdowns. We have developed methods of studying programmers’ work in order to deter- mine how the programming language and environment might be changed to prevent breakdowns. We have recently focused on preventing breakdowns in debugging. In order to see what tools might be useful, we per- formed two studies of both experts’ and novices’ pro- gramming activity [6] using the Alice programming environment (see Figure 2). We chose Alice for these studies because it simplifies the creation of programs by using drag-and-drop to place tiles of code into the code area and pop-up menus to choose parameters. This interaction prevents all type errors and syntax errors (see www.alice.org for more details). We observed that all of the programmers’ questions at the time of failure were one of two types: 32% were why did questions, which assume the occurrence of an unexpected runtime action, and 68% were why didn’t questions, which assume the absence of an expected run- time action. Furthermore, 50% of all errors were due to programmers’ false assumptions in the hypotheses they formed while debugging existing errors, resulting in the
insertion of new errors that had to be debugged.
Design of the WhyLine No existing programming environment allows users to ask these kinds of “why” questions. By analyzing the control flow graph of the programs, and annotat- ing it with the complete history of all assignments and uses of properties’ values, we are able to answer these questions directly in the Alice UI (see Figure 2) through the “Why did” and “Why didn’t” menus. The submenus contain the objects in the world that were or could have been affected. In addition to highlighting the relevant piece of code, we present a visualization of the answer to the question in Whyline— the Workspace that Helps You Link Instructions to Numbers and Events (see Figure 3). We performed a user study comparing the Alice environment with and without the Whyline. Subjects were Master’s stu- dents with programming experience rang- ing from beginning Visual Basic to extensive C++ and Java. Analyzing six sit- uations that were identical across the two conditions, the Whyline significantly decreased debugging time from an aver- age of 155 seconds per bug down to 20 seconds, which is a factor of 7.8. Furthermore, in the 90 min- utes allotted, programmers with the Whyline com- pleted 40% more tasks than those without. Full details are available in [7].
Conclusion We are currently working on extending these ideas in many directions. We are investigating new domains in which to design more natural languages. We are extending the programming environment research to help with other parts of program analysis and creation. While making programming languages and envi- ronments more natural may be controversial when aimed at professional programmers, we believe it is of significant importance for end-user development. In addition to supplying new knowledge and tools directly, the human-centered approach followed by the Natural Programming project provides a model of a methodology that can be followed by other devel- opers and researchers when designing their own lan- guages and environments. We believe this will result in more usable and effective tools that allow both end- users and professionals to write more useful and cor- rect programs. c
COMMUNICATIONS OF THE ACM September 2004/Vol. 47, No. 9 51
Figure 3. The Whyline’s answer shows a visualization of the runtime actions that prevented Pac from resizing.
