Download Verifiable Secure Operating System Software and more Lecture notes Operating Systems in PDF only on Docsity!
Verifiable secure operating system software*
by GERALD J. POPEK and CHARLES S. KLINE University of California Los Angeles, California
INTRODUCTION
While the desire for reliable security in multiuser computer systems has grown significantly, the computing community's knowledge of how to provide the requisite protection is still inadequate. Security is a "weak link" phenomenon; one link whose condition is unsatisfactory is the operating system software. It has often been pointed out that currently "no protection system implementation of any major multiuser computer system is known to have withstood serious attempts at circumvention by determined and skilled users."^1 The community is replete with apocryphal claims of secure sys- tems that inevitably have failed. Out of these difficulties and concerns has grown a good deal of activity. One part of the community has addressed the questions of how a system ought to be modularized, what its primitive elements ought to be, and what logical structure would be most useful. The useful concepts of capabilities and domains^2 have come from that activity. The Multics and Hydra systems are current examples of ongoing complex software systems in which serious attempts have been made to carefully design and structure the soft- ware with respect to protection considerations.3, Despite the value of these concepts and empirical labora- tories, our knowledge concerning the reliability of protection systems is disturbingly inadequate. Currently, it is not pos- sible to provide a meaningful guarantee that a system actu- ally provides the controlled protection which the design claims. It is not possible to state with assurance that clever users will be unable to circumvent the controls, and thereby gain access to information, operations, or other resources which the design intended to prohibit. What is required is a system which can do more than resist attacks by penetration teams. One would greatly prefer software which in a real sense has been proven correct with respect to certain precisely stated security predicates. Such a result would provide software protection of high quality, and help relieve fears that some feature or flaw had been overlooked. At UCLA, a multiuser computer system is being con-
- This research was supported by the Advanced Research Projects Agency of the Department of Defense under contract number DAHC- 15-73-C-0368.
structed in which it is expected that verification of the security properties of the software will be successfully performed. That is, meaningful and demonstrable operating system security is within our grasp. While the system is not yet complete, work has progressed far enough that the viability and quality of the resulting system and proofs seem assured. In the following, we discuss concepts which have contributed to the system's design, as well as issues that arose during that design process. It is expected that the approaches described here have more general applicability. Note that this work only concerns the protection enforce- ment mechanisms of the operating system software. Properly functioning, essentially error free hardware is assumed. The problem of authentication, that is, reliable identification of the user who presents himself to the system, also is not considered, except that a suitable environment is provided in which an authentication procedure could easily operate. In addition, the flavor of protection toward which this work is initially directed is fairly simple. Mechanisms to support mutually suspicious subsystems, memoryless pro- cedure calls, inference, or control of statistical access are not explicitly considered. Nevertheless, the insights provided by the goals of isolation and limited sharing apply to the more complex needs. We now discuss a number of thoughts found relevant to the design of secure operating system software.
DEFINITIONS OF SECURITY
Before continuing to discuss the general facets of the UCLA design philosophy, it will help to explain a distinction we make between security and viability. We believe oper- ating system security involves a set of essentially negative constraints. One desires to verify that certain actions cannot occur. For example, it will not be possible for the process associated with user x to have loaded with it a segment as- sociated with user y. It will not be possible for users of class x to access files of class y. One can coherently argue that such a point of view is incomplete; the null system satisfies negative constraints vacuously. What is required, the argument might continue, is inclusion of the idea of viable productivity—that the system actually supports useful activity. For example, if a
146 National Computer Conference, 1974
user could cause the process scheduler not to run any pro- cesses, would this not be a security flaw? This point is well taken. Certainly productivity is of importance since it em- bodies the primary reason for the existence of the system. Nevertheless, we argue that a meaningful distinction can be made between the prevention of undesired actions and the need for productive activity. To have verified the negative constraints is a useful and nontrivial step, and those con- straints certainly contribute to the overall viability of the operating system. These negative constraints express the security policy. For each user, we can translate these negative constraints into a list of which security objects this user should and should not be able to access and with what access types. Security objects are the physical and logical parts of a computer system that need to be controlled, protected, or whose status needs to be guaranteed. An incomplete list of examples in- cludes terminals, communication lines, processes, and files. Security policy may be expressed in terms of accessible sets. For user A, acces$ible[t,A] is the set which defines what objects this user is to be allowed to access at time t. All other accesses are to be prohibited. Each entry in the set is an object, access type pair. For each active object D (device or CPU), we define access[t,D] as the set of accesses (object, access type pairs) that object D makes at time t. We also define ovmer[t,T>] as the user on whose behalf the device is performing accesses at time t. For example, the CPU always acts for some given process. Then protection is enforced if ¥ t VD acce$s[t,D] is a subset of accessible[t,oumer[t,D]] We say that a system is secure if
(1) accessible sets describe the desired security policy and (2) protection is enforced.
The preceding is a simple conception of security. Neverthe- less, it can yield meaningful protection and a comfortable user environment as we demonstrate below. Such questions as whether user A can steal user B's files are, for example, directly handled by this model. Note that point one above contains aspects which will not be mathematically verifiable, since what is involved is a mapping to precise specifications from the intuitive notions which express user desires. One can check internal consistency of the resulting specifications, however.
THE CONCEPT OF AN OPERATING SYSTEM
SECURITY KERNEL
The value of segregating operating systems code into more or less disjoint layers has been clear for some time. One interesting question that presents itself concerns what the lowest level of the operating system ought to contain. De- signs focussed around message handling, process primitives, and others have been suggested. Recent efforts concerning the "nucleus" of the operating system are illustrated by Brinch Hansen^5 and Wulf.^4 There is still considerable debate
over this question, and its importance is not likely to decrease as systems grow more complex and additional layers, such as more sophisticated data management code, are added. It is our contention that the very lowest system level, the subnucleus, ought to contain the security relevant por- tions of operating systems primitives, and nothing else. The subnucleus that contains this isolated code we call the security kernel of the operating system. Note that such a design differs widely from current practices, as illustrated by OS/360 in which that code is strewn throughout the operating system. It has been pointed out many times that security is a crucial, basic quality of today's complex systems. The value of carefully designed modularity in the structure of operating system code is also clearly recognized. These two thoughts conjoin; enforcement of the intended modularity by security controls is of great aid in developing and understanding the system software. It is easier to obtain reliable enforcement of modularity if the proper functioning of those mechanisms which provide that enforcement does not depend at all on other code. That is, modularity enforcement code should depend on as few and as simple modules as possible. If security is basic to a system, then the code which provides security should also be basic, with few dependencies, hence at the lowest levels of an operating system. It is exceedingly difficult to understand the interactions of security relevant software when the code and the implications of the design decisions which produced .that code are dis- tributed throughout the operating system. The difficulties arising from this distribution are illustrated by the large number of known security flaws which involve the interaction of a number of characteristics of an operating system design and implementation. As one simple example, in a number of systems, the code which checks a security condition and the code which takes the action guarded by that check are separated by some distance in the execution sequence. It is possible under certain conditions, an interrupt perhaps, for the user to change the parameters examined by the check before the action occurs. Such flaws usually depend on the availability of parameters to user code and the interrupti- bility of the check-action pair.^6 As the subleties of these interactions are seen, the near hopeless quality of poorly planned and distributed security relevant code becomes more and more clear. The isolation and centralization of security relevant code provides more than merely a better basis for understanding that code, important as that understanding is. By isolating that code at the heart of a system, running on the bare hardware, its correct functioning does not depend on the proper behavior of other modules of operating system soft- ware. This fact provides several advantages. Outer layers of the software may be written and modified without need to reevaluate the security of the entire system. Maintenance is then eased. In addition, the resulting security kernel as a relatively small, isolated, independent set of programs is susceptible to a formal verification of its correctness with respect to security. The importance of this property is diffi- cult to over-emphasize.
148 National Computer Conference, 1974
the virtual machine environments. Thus, construction of a VMM is a relatively simple and cheap way to obtain a clean interface for programs, and the amount of effort required to demonstrate the utility of the kernel is decreased. Such an approach involves protecting and controlling entire virtual machines. Hence two users running under an oper- ating system in a single virtual machine are accorded no protection from one another by the security kernel, although they are protected from other virtual machines. The obvious solution to this problem is to run each user with his own operating system in a separate virtual machine, using execute only sharing of common operating system code to maintain efficiency. A second advantage of a virtual machine approach how- ever is more intrinsic, but also accrues in part from its simplicity. Earlier, it was demonstrated with the example of bulk I/O that, depending on one's conception of the kernel, it was potentially possible for higher levels of software to misuse kernel primitives in a fashion that could lead to a security flaw. This problem is diminished if the higher levels of software are simpler. Inter-segment linking, access control over spooling pro- cesses, and many other operating system features are absent in a virtual machine monitor. The complexity of the "pro- tection semantics" associated with aspects of operating systems such as those mentioned above, as well as the necessary supporting code, can easily lead to security errors. By contrast, a virtual machine system projects a very simple environment from the point of view of security. Sharing may be provided by very simple methods while still main- taining the richness of an individual user's environment. This relative simplicity makes it practical to demonstrate that there do not exist interactions of the sort mentioned earlier which cause security flaws. Essentially, there are many fewer features to interact.
COMMUNICATION CHANNELS
Another relevant view of security deals with the control of communication. For example, a system for which a user with one clearance could not pass certain data to a user with a lower clearance no matter how hard both try would be of value to the military. This problem essentially is one of showing that there does not exist a communication path between two processes. Up to this point, we have been con- sidering a fairly specific definition of security, albeit one that can involve fairly subtle interactions of code. However, as pointed out by Lampson,^10 there are a number of con- ceptual channels of flow of information in a shared system. First, there is the obvious one that most people speak of in the context of security, the explicit passing of files and other units of information, reading another user's work space, and the like. But there are others, often involving the passing of re- sources—assigning and deassigning devices, or making service demands on the system that other users can sense. Exactly what channels exist, what their respective potential band-
widths are, and what mechanisms and costs are needed to seal them are all questions that need to be resolved in order to decide how these communications channels should be handled. Of these channels, we first distinguish between those which depend on timing considerations, and those which do not. A timing dependent case is demonstrated by process A sensing the CPU load of the system caused by process B. In the context of a virtual machine system, one way these channels may be blocked, at least in so far as the running programs are concerned, is by properly simulating the passage of virtual time to VMs, and not providing a real time measure. Timing independent channels also exist, however. Con- sider the following case. Let PI and P2 be separate processes who are not to communicate. The devices D l and D2 are discs controlled by a common scheduler S. Requests for I/O may be considered as messages from PI and P2 to S, and completions as messages from S to PI and P2. Notice that in this example, both P i and P2 are using both devices. In keeping with the kernel design philosophy, it is desired that S be kept out of the kernel. How then are P i and P prevented from communicating via the scheduler S? Both send S information, and both receive information from S. Are we forced to prove assertions about S? If our system already existed, its design were as above and we were at- tempting to secure it in retrospect, without major redesign, the answer is likely to be yes. Instead, let the kernel do all the moving of information, and treat the messages as being contained in "locked boxes." A scheduler may read (but not change) such a locked box. Label messages from P i and P2 with their destination, D l or D2. The return messages from devices are similarly labelled from either D l or D2. The scheduler now merely queues requests and hence cannot change message contents.
SOURCES, SINKS, AND THE MORSE CODE
PROBLEM
The above design does not completely block interprocess communication, for a "Morse Code" mechanism still remains. That is, both PI and P2 can pass S two types of tokens that S can distinguish: D l type and D2 type. Hence S can receive an arbitrary binary encoded communication from each of the two processes. Furthermore, S can send a nearly arbitrary message back to each process by ordering the return of completions (nearly arbitrary only because he may run out of tokens of one or the other type or both). We have been had by a malevolent scheduler. Skeptics should remember that binary codes are the charwomen of contemporary computing. This problem however is not intrinsic to security systems nor an inherent defect in the kernel concept, but rather merely one of inappropriate design. Split the device scheduler S into two schedulers, Si and S2, one for each device. Now each scheduler, if we continue to operate in "locked box" mode, deals in only one variety of token, and hence there is no binary communication channel.
Verifiable Secure Operating System Software 149
The preceding is a rather attractive solution. The sched- ulers SI and S2 need not be proven at all, and they may have full access to information necessary for scheduling. Little compromise with respect to performance has been made. The two schedulers may even share common code, as long as their writable storage is separated. The task of providing proven security has been eased, with the only important cost having been careful design. The above solution is valid because Si and S2 are purely information sinks. While they receive message contents, there is no way for them to broadcast, so long as only one request per process per scheduler may be pending at any time. They only request that the kernel return device status to the processes. More generally, the solution is effective because the flow of information with respect to Si and S2 is one way. In this case, the two are sinks. It would have been equally sufficient with respect to security if they had been pure sources. Of course in this case that would not provide the desired func- tionality. The principle here that does generalize usefully is that one can reduce the security problem significantly by first isolating as much as possible, often in ways that are not immediately apparent. This principle has been applied to the UCLA kernel a number of times, the device schedulers and virtual machine simulators being two examples. At this point one might argue that these subtle channels are in certain respects irrelevant because they seem to require the active, coordinated cooperation of all the parties involved in the communication. If two users wish to communicate, let them; isn't that the strength of multiuser systems any- way? However, in the disc scheduling example raised earlier, even without process Pi's cooperation, it would have been possible for P2 to learn about Pi's I/O characteristics. More important, security may be compromised by incorrect high level design. Errors in outer modules, such as the scheduler, are exploitable. Careful proof procedures should discover these cases.
VERIFICATION
I t is important to realize that even if no security verifi- cation were actually performed, the design philosophy de- scribed here and the discipline imposed by the intent to verify, together, have already increased the reliability of the security aspects of the system. Nevertheless, there is more which can profitably be done; verification is an attainable goal. Program verification tools are not yet in a well developed state. The largest program that is known to the authors to have been verified to date contains somewhat more than two thousand instructions; small compared to most operating systems.^11 Hence, it is necessary to severely limit the size of the program about which one wishes to prove certain prop- erties. The UCLA kernel is approximately 500 lines of high level code. We are in the process of verifying that code and expect to be able to complete this task with a reasonable amount of effort.
It is important to realize that program verification is not the whole of operating system security, although it is a very important part. Verification establishes the consistency of a program with a separate, essentially static statement of what that program is, or is not, to do. In order to apply verification, one first needs explicit definitions of security, and those definitions must be translated, currently by hand, into predicates that may be handled by mathematical means. Second, one is faced not by a large flowchart, but rather by a number of kernel primitives, which potentially may be invoked in any order. In addition to verifying certain prop- erties of the primitives themselves, it is necessary to demon- strate that there is no order of invocation of primitives that would result in a security violation. If the primitives are thought of as composing an action space, then one needs to demonstrate that there does not exist a path through that space, the result of which would make one of the security predicates not true. There are a number of strategies which can make this proof process easier. After carefully categorizing every inde- pendently addressible object and action to make the model complete, naming techniques can be employed to segment the proof task. If we can demonstrate that objects only have a user's name associated with them if they are members of his accessible set, then we can show protection enforcement by showing he can only access objects with his name. As a second proof strategy, certain required predicates may be included as run time checks. For example, in a military system, one might wish to guarantee that a user with secret clearance is not able to access a top-secret file. One way to guarantee that constraint is to embed a run-time check in the (only) I/O routine. All of these considerations help to make the verification task possible although far from easy. They also support the undesirability of ex post facto verification.
THE UCLA-VM SYSTEM
Let us give some examples of the previous remarks from the UCLA-VM system. A sketch of the structure of the UCLA virtual machine system is shown in Figure 1, with the VMM broken into its attendant parts. The objects in the system have been specifically defined and are homo- geneously treated and viewed by the kernel. That is, the kernel has no knowledge of the internal structure of any security objects with the exception of the protection data. This data is the basis for security decisions, and plays a role analogous to the contents of the Lampson-Graham-Denning protection matrix.^12 It is packaged into security objects so that access to the data may be controlled. Thus, control is obtained over the way protection decisions are changed. The blindness of the kernel to other objects' internal structure simplifies matters, but it implies that only actions among objects are monitored; no intra-object control is provided. As a result, activity within a virtual machine is not controlled by the kernel. Two users running in a single virtual machine are accorded no protection from one another by the kernel.
Verifiable Secure Operating System Software 151
We will also estimate what portion of the observed overhead is due to virtualization costs. Hence, we confidently expect to demonstrate that a simple, but useful form of verified multiuser security may be obtained at a quite acceptable cost.
CONCLUSIONS
This work has been intended to demonstrate several points. First, and perhaps most important, it is practicable to have verified software security in multiuser computing systems. Second, the approaches of kernel design, virtual machine monitors, and mathematical verification of the properties of software contribute usefully to the task of providing verifiable security. Nevertheless, a great deal remains to be done. It has been and continues to be a taxing effort to obtain this high level of software security for any given system, in part because our tools and concepts are still unrefined. These facts are encouraging, for they suggest that considerable progress can be expected. Furthermore, the system described here provides only limited sharing, and does not, for example, address the problems of mutually suspicious subsystems or memoryless- ness at all. It has been argued that multiprogramming of resources, rather than information, is still the predominant activity, at least among the security conscious segment of the computing community. Although this segment would be satisfied by a virtual machine approach, there remain vital activities for which reliable control of sharing is crucial, and those activities are not expected to decrease in the future.
APPENDIX
The security kernel and virtual machine monitor are being constructed for a DEC PDP-11/45 that is being attached to the ARPANET at UCLA. The PDP-11/45 has a three state architecture which naturally lends itself to the needs of the kernel, VMM, and user environments. However, the Unibus I/O structure of this machine does not lend itself conve- niently to virtualization, since nearly every instruction is po- tentially an I/O instruction and most be simulatable, unlike the limited set that normally accompanies machines with more conventional I/O processors. However, much more important than this inconvenience with respect to I/O is the fact that the standard PDP-11/ cannot be virtualized at all. In Reference 8, it is stated that hardware must have certain characteristics in order for a VMM to be constructible. Briefly, instructions that affect control of the processor, or whose behavior are disturbed by the presence of the VMM, are termed sensitive instructions. All sensitive instructions must be privileged in order that they may trap to the VMM to have their effect simulated. In the standard PDP-11/45, there are nine instructions for which trapping is necessary but which are not privileged. This fact makes the machine impossible to virtualize.
In addition, one would like trapping of instructions to be a function of the mode in which attempted execution of the instruction occurred. The reason such behavior is desirable is a result of the following considerations. It will be natural to run the kernel in the most privileged mode, the virtual machine monitor in the next most privileged mode, and the virtual machines in least privileged mode. One then prefers that instructions which trap in a virtual machine be re- flected by the hardware to the VMM, while it may be necessary that the same instruction executed by the VMM trap to the kernel. Such mode dependent trapping has been suggested before.^15 In the case of recursive virtualization, in CP-67 for in- stance, this behavior is simulated by the software. The hardware traps all instructions to privileged mode, and soft- ware reflects some of them out. Here, however, there is an additional motivation. The existence of mode dependent trapping makes it unnecessary to have the reflection software in the kernel. It needn't exist at all. As emphasized in the body of this paper, the need to exclude non-security code from the kernel is almost as important as including all the relevant code. The DEC/UCLA hardware modification pack- age also includes other features for efficiency and/or con- venience.
REFERENCES
- Schell, R., Private communication, USAF/ESD L. G. Hanscom Field, September 1972.
- Lampson, B. W., "Dynamic Protection Structures," Proceedings of FJCC, 1969.
- Saltzer, J., "Protection and Control of Information Sharing in Multics," ACM/SIGOPS Symposium on Operating System Prin- ciples, Yorktown Heights, October 1973.
- Wulf, W., HYDRA: The Kernel of a Multiprocessor Operating Sys- tem, Carnegie-Mellon University, 1973.
- Brinch Hansen, P., "The Nucleus of a Multiprogramming Sys- tem," CACM, April 1970.
- Bisbey, R., Private Communication, USC/Information Sciences Institute, June 1973.
- Buzen, J. P. and U. Gagliaridi, "The Evolution of Virtual Machine Architecture," AFIPS Conference Proceedings, Volume 42.
- Popek, G. J. and R. Goldberg, "Formal Requirements for Virtualiz- able Third Generation Architectures," Communications of the ACM, Vol. 17, No. 7, July 1974.
- [IBM CP-67] IBM Corporation, Control Program-67/Cambridge Monitor System, IBM Type III release No. 360D-05.2.005, IBM Program Information Department, Hawthorne, New York.
- Lampson, B. W., "Dynamic Protection Structures," AFIPS Con- ference Proceedings, Volume 35.
- Ragland, Larry C , A Verified Program Verifier, Ph.D. Thesis, University of Texas, Austin, 1973.
- Graham, G. S. and P. J. Denning, "Protection—Principles and Practice," AFIPS Conference Proceedings, Volume 40.
- Roberts, L. G. and B. D. Wessler, "Computer Network Develop- ment to Achieve Resource Sharing," AFIPS Conference Proceed- ings, Volume 36.
- Kleinrock, L. and W. E. Naylor, On Measured Behavior of the ARPA Network, to be published.
- Goldberg, R. P., "Architecture of Virtual Machines," AFIPS Conference Proceedings, Volume 42.
