Word Meaning Selection: Surface Semantics and Contextual Disambiguation, Papers of School management&administration

Download Word Meaning Selection: Surface Semantics and Contextual Disambiguation and more Papers School management&administration in PDF only on Docsity! IEKE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMt.6, NO. 4, ,“LY ,984 493 Word-Meaning Selection in Multiprocess Language Understanding Programs RICHARD E. CULLINGFORD, MEMBER, IEEE, AND MICHAEL J. PAZZANI ,ibsrmc:-An understander readinS or listeninS b, sanmne speak hrs to repeatedly solve the problem of wad-meaning ambiguity, the s&c- tion of the intended meaning of a word from the wef of its posibte meminp. For cxmnpke, the pmblem of pronominnl reference can be mnsidered as a cboosinSof the intended refemnt from the cotIectiori of entities which have Jmady been mentioned or which cm be inferred. Human understanders apply rides oi syntax, surface semmtics,gFnernl world knowledge, and vartous types of contextti knowled~ to m. sdve wordaense or prmwmind ambiguity as they prweu tmguage. We describe a mechanism, catted I coopemtive word-mawing selerfor, which attows the computer to we tious knowledp sources as it %I- derstands” text. The wordmc.rdnS &ctor i, put of P conceptual malyzer which forms the lutullsngua~ interface for a paix of multi. prctes language processing systems. The fust, alted DSAM (dis- tributable script applier mechanism), reads and summarizes newspaper articles making heavy reference to situationat scripts. The second, ACE (audemtc munaeltng experiment), is a converutionll pm- which wtomntea certti parts of the academic counseling task. In each Of these system*, . variety of knowted&+ murces, each manrged Ly a disfinct **exp&” process, is broug#,t to bear to enable the word- meaning selector to form tbc mart plnusible reading of P sentence con. tabtinS ambiguous words. Index Terms-Distributed understanding, natural la,,g,,agc processing, reference, word-sense disambiguation. I. INTRODUCTION I” N his noted paper describing what has come to be known as the “Tunng test” (371, the British mathematician A. M. Turing proposed to’ answer the question “Can a machine think?” in terms of its ability to play a certain “imitation game.” The game has three players: a man, a digital computer, and an interrogator in a room apart from the other two. The interrogator is supposed to decide, on the basis of questions Put to the two players, which is the mm and which the com- puter. Questions and answers are communicated back and forth over a teleprinter. If the computer, by imitating human- like responses to tithe interrogator’s questions, could lead the interrogator to decide wrongly in the same percentage of cases as when the imitation game is played between, say, a man and a woman, we would conclude that it was indeed “thinking” in a truly human fashion. The point of separating the interrogator and making him Manuscript received March LO, 1982:revised November 15.1982 and l”tY 25, 1983. This work was supported in part by the Advanced Re- *arch Projects Agency of the Department of Defense, and monitored bY the Oftice of Navlf Research under Contract NOO14-79-C-0976. R. E. Cullingford is with tbs Department of Electrical Engineering md Computer Science. University of Coniwticut, Storrs, CT 06268. M. 1. Pazrani in with the MITRE Corporation. Bedford, MA 01730. talk to the others over a teleprinter is, of course, that thinking is an intellectual activity having little to do with physical appearance or communication apparatus. Moreover, the question-answering format of the imitation game is a suitable means of focusing on the various capabilities that are normally considered to require thinking. For example, the interrogator could ask the computer to write a poem, play chess, or solve an algebra word problem. There is, however, a prior problem, to be solved before the computer can demonstrate its ability to perform intelligently In tasks such as these. To engage in a question-answering session it must be able to conduct a conversation with the interrogator. If it cannot imitate the conversational expertise of the human, it will never succeed at the Imitation game. The thinking computer, therefore, must be expert at natural. language communication. This ~eerns to require three distinct but strongly interdependent processes: the ability to analyze a textual input to get at its meaning content (cf. [16], (271, [36], [38], [39] ); to make inferences about the components of meaning that are only implicit in the input (cf. [4], [S] , [8] , [26], [33] ); and to generare appropriuze responses based upon the current input and conversational history (cf. [2], ill], WI, 1221). This paper discusses a key problem in the computer analysis of natural language, the selecrion of intended word meanings in contexi. Section II presents the problem in terms of the class of programs known as conceptual analyzers, and describes knowledge sources needed for word-meaning selection. In Section III, we propose a partial solution to the problem, embodied in a computer algorithm called a coopemtie word- meaning selecror. The algorithm provides a mechanism whereby alternative word meanings can be explicitly manipu- lated by various memory processes as these attempt to cooper- atively home in on the most plausible &dings of sentences. The algorithm is part of a conceptual analyzer which maps natural-language strings into a conceptual dependency repre- sentation [31], [32] of their meaning. The analyzer, which is based upon the one described in [l] , forms the natural-lan- guage interface for a pair of multiprocess text “understanding” systems. The fast, called a distributable script applier mecha- nism (DSAM) [IO], reads and summarizes newspaper stories making heavy reference to situational scripts [33]. The second, ACE (academic counseling experiment); is a conversational program currently being developed to automate certain parts of the academic counseling task [12]. The Appendix gives annotated output from the word-meaning selector as it runs during a story-understanding task. 0162.8828/84/07000493$01 .OO 0 1984 IEEE II. CONCEPTUAL ANALYSIS AND WORD-MUNING SELECTION A. Conceptual Analysis The machine’s ability to understand its input is a key~com- ponent in any automatic natural-language processing task. Hence, language analysis has been the subject of considerable research attention in artificial intelligence (AI) and computa- tional linguistics. The model of understanding we wish to describe in this paper is connected with the class of language- input program called concephul analyzers. Analyzers of this kind are designed to map an input string directly into a representation of the meaning of Ihe string, using whatever morphological, syntactic, semantic, contextual, etc., cues are available. Conceptual analyzers normally operate from left to right, In one pass, a lexical or phrasal unit at a time. Their output is stated In terms of the conceptual repre- sentation system used by Inference and memory-search pro- cesses (e.g., [30], [31], [39]). Ideally, ‘tiell-formed” com- ponents of this output (called “conceptualizations” [31]) are made available to the memory functions as they are formed from the input stream. Conceptual analyzers are distinguished from other types in that they do not attempt to first analyze the Input syntactically, then assign a semantic reading to the syntactic Structure (see; e.g., [6], [14], [19], (251, [41]). Nor do they conduct a simultaneous syntactic and semantic analysis (cf. [3], [40]). (These latter types of analyzer are often called “parsers.“) Syntactic features such as word order and noun-group con- stituency are used by a conceptual analyzer only to guide the conceptual mapping process. B. Problems in S&wing Word Meanings The most formidable problem faced by a conceptual analyzer is that of word meaning selection. The phrasal/lexical units that it sees in the input are usually analyzable into more than one meaning Structure. If the analyzer defers choosing a repre- sentation, the number of possibilities grows as the product of the number of individual readings, and the analysis process soon gets out of hand. Therefore, as people seem to, the analyzer has to decide on a representation as quickly as possi- ble, even at the risk of having to backtrack because of a “garden path” input. The best-known case of the meaning-selection problem is word-sense disambiguntion. A word sense is a distinct mean- ing of a word (found, e.g., under its dictionary entry), with a distinct underlying representation. Consider, for example, the following usages of “sense”: (la) Most words have more than one sense. (lb) Sight is our most important sense. (lc) Some people have no sense at all. “Sense” is a well-behaved word In that it has only a rela- tively few alternative meanings. Other words have dozens of possible readings. For example, “give” and “take” have been metaphorically extended to so many situations that they are essentially meaningless in isolation. Their disambiguation requires access to substantial amounts of context. A computer word disambiguation scheme, therefore, will require a model of context consisting of both the meanings of surrounding words and higher level expectations. Choosing the referent of a definite noun phrase is another example of the word-meaning selection problem. A definite noun phrase consists of a definite name, a pronoun, or a co”. struction introduced by the definite article or certain types of modifiers: (2a) John kicked the ball. (2b) The Celtics extended their streak. (2~) He threw John out. The problem here is choosing the real-world referent ofphrases such as “John,” “ihe Celtics,” “the ball,” “their streak,” and “‘he.” Notice that the choice of referent often interacts with the word-sense disambiguationprocess. For example,memory’s ability to identify a real-world ball in John’s vicinity reinforces the selection of “round toy” as the intended meaning of “ball” in (2a). Example (2b) Illustrates the process of fading a r&r. ent in the current clause unit (i.e., identifying “Celtics” with “their”), which in turn reinforces the reading of “streak.” Finally, the identification of “he” as a bartender in (2~) gives a different meaning to the sentence than if “he” were a third baseman. A fmal example of the word-meaning selection problem occurs in ellipsed inputs. These are sentence fragments (often noun phrases) presented without their accompanying proposi- tions, most often during a conversational interaction. Far exsmple, in: (3a) Q. Where did you go on New Year’s Eve? A. 3 parties. (3b) Q. Who’s eligible for Federal matching funds in the ‘80 election? A. 3 parties. reference to the conceptual form of the immediately preced. ing question is needed to select the intended sense of “parties.” C. Knowledge Sources for Word-Meaning Selection Clearly, to solve the word-meaning selection problem the computer will have to be given various sources of knowledge about natural-language phenomena and means for applying the knowledge as appropriate. This section contains a brief discussion of some kinds of knowledge that seem to be needed, as an Introduction to a conceptual analyzer capable of certain kinds of word-meaning selection. The simplest, and probably best understood, knowledge source for this task is rules of syntax. The intended reading of ‘Wsiting” in the following example cannot be determined. without examining context: (4a) Visiting relatives can be nuisance. If we change the syntactic form of (4a) slightly, however, the meaning is clear: (4b) Visiting relatives is a nuisance. (4c) Visiting relatives are a nuisance, C,j,,L,NGpORD AND PAZZANI: WORD-MEANING SELECTION 497 ACE: ACE: You will have to take Math 134 next semester. You may take CS 111, CS 207, a group 2 course, and a group 3 course. fore, the main line of development in CD analyzers has been the attempt to incorporate more and more context into the analysis process. The basic idea here is to set up a course Khedule for the student which makes sense In terms of the student’s current $tauding. TO do this, ACE needs to know what the student is taking now. The first question in the dialog above is designed to elicit this information. Since engineering students typically take four or five courses a semester, the response to this ques- tion is incomplete. ACE’s understanding of the ellipsed answer depends upon the analysis of afnbiguous terms such as “com- puter science” in the context of the question which WAS asked. On this basis, “computer science” is taken as referring to a couise rather than the field or a” academic department. Even so, the response as given indicates several problems. First, there are several ways to complete the undergraduate chemistry requirement. One way is to take the sequence Chem ;?9/Chem 130. It is highly unusual for a first-semester fresh- man to be taking Chemistry 130 since it has a prerequisite. In some cases, a student may be able to get advanced credit by passing an examination based on the prerequisite’s subject matter. ACE is “aware” of this possibility as it asks the second question. Word definitions describing the meaning structure(s) built by a word and suggestions for using this structure are typically kept off-line in a dictionary, and are not called into active memory until the word is actually seen in the input stream. Expectations associated with a word defmition are encoded in a special type of production (or test-action pair [21], 1241) called a request [27] . Requests are activated when the associated word deftition is loaded. The activation process places the requests in a short term memory of requests to be considered. Request consider- ation repeatedly selects a request and evaluates its test part. If the test is true, the request is said to have “fired,” and its action part is executed. Having solved the problems caused by the answer to the first question, ACE attempts to fmd out what other courses the student is taking. It notices that the highly expected mathe- matics course sequence has not been mentioned, and immedi- arely tries to find out why. The curriculum specifies that Math 133 is the expected course at this point, and this deter- mines the form of the question. Because the question explicitly mentions Math 133, the referent of “it” in the response “I passed it in high school” can be supplied to the analyzer. This in turn dlsambiguates “passed.” Having determined the student’s current course load to its satisfaction, ACE then consults the curriculum again to find out which courses are mandatory at this point, and which are o:itional. Note, however, that the responses it generates at tile end of the dialog are critically dependent on its under- standing of what was said before. Requests can check semantic, lexical, or contextual features of the runtime environment, and create or connect conceptual dependency structures. Moreover, they can cause other re- quests to be loaded or deleted. Associated with the meaning structure built by a word (sense) are a set of roles and a set of expectations embodied In requests indicating how the roles are to be fdled. Consider, for example, the sense of the verb “to take” which means “to execute the academic-course script” from the point of view of the student. (To “give a course” or to “teach a course” is to execute the same script from the point of view of the teacher.) In a~ simple English format, the requests asso- ciated with this sense of “take” would be: REQUEST1 : TEST: Is the “object” of take a course? ACTIONS. Create the concept for a” execution of the course script. Fill the conceptual object slot of this concept with the course that was found. Activate REQUEST2 REQUESTZ: TEST: III. AN ALGORITHM FOR WOAD-MEANING SELECTION Having sketched some important analysis problems posed for systems such as DSAM and ACE, we now turn to a more detailed discussion of how these problems are solved by the Conceptual analyzer they use. A. Conceptual Dependency Analysis Language analysis in the conceptual dependency (CD) para- digm, motivated by the way that people seem to approach the task, has attempted to use predictions or expectations about what will be heard as the driving force behind the under- standing process.s Syntactic, surface semantic, scriptal, and Planning contexts are all rich sources of predictions. There- aThe arguments for predicthe understanding, and for conceptual analysis in general, xc covered in detail in [271, (281, [341. Is the “subject” of take a person? ACTIONS. Fill the conceptual actor of the course script with the person that was found. These requests contain two different types of information. ‘Positional” specifications predict where in the sentence the conceptual actor and object of the take-course script *Ill be found. For example, the “object” spot in REQUEST1 Is the syntactic object of the clause containing “take” if the sentence has the active voice, the syntactic. subject if it is passive. ‘Se- mantic” specifications constrain the entities that will be used to ffi the conceptual actor and object role In the take-~ourse concept. Existing CD analyzers such as ELI (English language inter- preter [ZS]) and CA (conceptual analyzer [I]) use the test part of the request to implement a form of word-senw dis- 498 IEEE TRANSACTIONS ON PATTERN ANAiYSlS AND MACHfNE INTELLIGENCE. “0‘. PAM1.6. NO. 4. JULY 198‘. ambiguation based on surface semantics. For certain words, a group of tests checking lexical, syntactic, or semantic features can be used to determine which sense is Intended. In such a case, the tests are said to be orthogonal, i.e., the tests check mutually exclusive cases and cover all possibilities in such a way that exactly one request Is tired. The action of the request which has fired will create the meaning representation for the intended word sense. In this manner, one word sense is selected and the others are suppressed. So, for example, we could add a second word sense to the deftition of “take” corresponding to a sentence such as “John took a” aspirin” in the following way: REQUEST3: TEST: Is the “object” of take a drug? ACTIONS: Suppress the other requests of “take.” Create Fe concept for a” INGEST action. Fill,the conceptual object slot of this concept with the drug that was found. are examples of language analyzers which implement proposed solutions to the problems caused by words with multiple senses. In ELI, expectations are used to choose among word senses. Requests associated with a word are activated only if their actions build a conceptual dependency stmcture which is expected by an already existink request. “Expected” here means that the test part of the existing request would become true if the meaning structure whibh the new request builds were added. (The process of extracting the meaning represen- tation from the action part of a request Is called rehearsal.) Note that this is a top-down approach. ELI matches meaning structures created (through rehearsal) by new input to its expectations. There are several problems with this approach, all caused by the requirement for a pm-existing expectation. Since conceptual entities must have been predicted before they are accepted Into the system, and since Initially there are no expectations, there must be a standard set of initiating requests at the beginning of each new sentence. However, in sentences such as: Activate REQUEST4 (to fmd a” actor, as above). This method of selecting word senses by using orthogonal requests works well for many verbs. The intended word sense is determined by the class of actor or object associated with the verb. The method is also useful in disambiguating some adjectives. For example, two meanings of “rich” can be dis cerned in “a rich ma”” and “a rich cake.” Here, the concep- tual type of the modified noun (person versus ingestible object) is enough Information to select the proper meaning of “rich.” Words which CM be disambiguated by orthogonal requests have a common feature. Their meaning is embodied in a case fmme of conceptual cases and fillers, with a request looking for a conceptual entity to fti each case. The type of concep. tual entity found can determine which sense is Intended. Nouns which build picture producers (311, on the other hand, do not have this feature since typically they build case frames with all the cases already ftied in. (Picture producers are c&n. cepts corresponding to entities such as persons, places and objects which tend to produce a mental image in the mind of the understander.) For example, in the sentence: (10) John shot two bucks. The pilot and co-pilot died, authorities announced, the second clause is not expected and the initiating requests are no longer active. EL.1 can properly handle the disambigua. tion of “ball” in: (lOa) John kicked the ball. (lob) John attended the ball. because “kick” activates a request containing the proper pre- diction. However, it cannot handle the passive forms of these sentences: “John” builds a picture producer for a male person named John. This concept is “complete” in the sense that it can be understood in isolation, as, for example, “shot” cannot. Sbni- larly, although the phrase is ambiguous in (lo), “two bucks” builds well-formed structures for either %nount of money” or “male deer.” It would be difficult, if not impossible, to define a set of orthogonal tests to select the intended mea”. ing of ambiguous nominals such as “buck.” The problem with this approach is that each word is responsible for disambigunt- ing itself. To select the intended sense of a word which CM create several different picture producers, expectations about the intended conceptual class must be used. B. Rekued Work in Word-Meaning Selection K&e&‘* ELI [32], Small’s word expertparser(WWEP) 1361 and Hi**t and Chamiak’s Polaroid words mechanism (PW) [16]’ (1 la) The ball *a3 kicked by John. (1 Ib) The ball was attended by John. because t@ expectation needed to disambiguate “ball” comes after the word. EL.1 does not have the ability to delay deciding among requests to activate. The word expert parser is a complex and interesting concep tual analyzer capable of performing in a bottom-up fashion sever! of the types &word-meaning selection considered here. I” WEP, each word comprising a bundle of w&d-senses is assigned a” Individual “word expert.” A word expert is represented as a coroutine which cooperates with neighboring words to select its intended manse and eventually to build a meaning structure for the entire sentence. (The meaning representation system is a variant of Rieger’s Commonsense Algorithm notation [26] .) A word expert’s basic mechanism for selecting one of its senses is a discrimination “et in which n-way discriminations (called muliiplechoice tests) can be made on the basis of lexical and semantic properties of neigh. boring words. ” The sources of knowlkdge which the word expert parser uses are surface semantics and, to a lesser degree, general world knowledge coded into a” Individual word expert. However, it apparently has no way of udng the higher level forms of pre dictions provided, for example, by scripts, plans, and discourse context. So, for example, it could not disambiguate ‘took” in: (12a) David was arrested because he took a bottle of aspirin. (12b) David died because he took a bottle ofaspirin: C”I,L,NGFORD AND PAZZANI: WORD-MEANING SELECTION 1” other cases, the Information requested by the multiple- choice tests of the discrimination net will not be present. For example, the semantic category of the object of “take” is needed to discern between “take a course” and “take medi- _ tine.” In example (13), however, this is not immediately available: : (13) David took it. Question from the word expert of “take”: What did David take? A method for performing pronominal reference has not been implemented in WEP. In fact, such a” algorithm could not use the standard word expert discrimination net mechanivn be- cause this requires the possible senses (or antecedents in the case of pronominal reference) to be known when the word expert is coded. The Polaroid words (PW) mechanism is a recent system for combining the processes of word-sense disambiguation and m&ing connections in a semantic network of scripts/frames. (The iatter process is also called “‘marker passing.“) PW assigns to each surface word a structure which contains all of its possi- ble senses, and conditions for the selection of one of the mean- ings. For example, the adjective “green” knows that its color sense can only qualify ~a physical object. PW “negotiates” among competing senses, most importantly by examining the length of connection chains established by marker passing among the preceding words. PW, in concept, is very similar to the VEL mechanism de- scribed below. Important differences include: 1) PW is not a” analyzer, per se, but cooperates with a syn- tactic analyzer based on [25] ; 2) since PW currently exists only In prototype form, it is difficult to determine whether it can be extended to handle the pronoun resolution and frame-selection prob- lems to be described below. C. A Representation for Multiple Word Meonings Existing conceptual analyzers, then, fail to handle many tiportant cases of word meaning selection. There are two main reasons for this: 1) the handling of expectations is con- strained too much, as in ELI, by the topdown nature of the control structure; or 2) as in WEP there is no way to make use of high-level expectations during disambiguation. Both ap- proaches suffer from the fact that the alternative senses of a word cannot be explicitly seen by the disambiguatingprocesses, being hidden in ELI, for example, inside the action parts of requests to be rehearsed. The approach taken in this work builds upon the design of Birnbaum and Selfridge’s CA, which implements an essentially, bottom-up approach to conceptual analysis. CA allows word definitions to add concepts and expectations to the analyzer’s short-term memory (called the “concept list,” or C-LIST) essentially at will. Thus, it avoids ELI’s excessively top-down nature. On the other hand, expectations embodied in requests are handled uniformly no mutter what their source. Thus, the opportunity exists to have memory processes examine and modify the current state of the analyds process. Two things are needed to make this work. One, we must 499 have a representational system for words with multiple meanings which makes the alternatives visible. Secondly, a uniform repertoire of procedures to manipulate the alternatives must be made available to the processes capable of making a selection. Our approach is implemented in a conceptual analyzer called APE (a parsing experiment), which extends CA to handle these more general kinds of meaning selection processes. We use the following simple declarative representation for a word with several word senses: (VEI_ VI “sense 1” v2 ‘~sense 2” . . . V” “Sense n”) where VEL (Latin “or”) indicates a mutually exclusive set of possible meanings. The dictionary defmitian of a word with multiple meanings always contains a request to add all its senses (i.e., add a VEL) to the analyzer’s short-term memory, the C-LIST. At the same time, a pool of requests may be activated to aid in the disarn- biguation of the VEL. Selection of the Intended component of the VEL follows these procedures. 1) The request creating the VEL may activate another re- quest to examine the C-LIST for a concept with a semantic feature, to check the input sentence for a lexical feature, or to query a script or plan applier or other memory module for a contextual feature. A request of this type could assert which meaning is intended or eliminate senses which are not Intended. 2) A pre-existing request may be looking for one of the possible meanings of the word creating the present VEL Typ ically, expectations of this kind come from surface semantics (e.g., “attend” expecting a” event as its object). 3) Most importantly, a VEL may be disambiguated by expec- tations explicitly set up because some sort of context has been established: *) b) knowledge structures such as scripts or plans; discourse contexts such as permeate narratives (which allow pronoun and other definite nominal references to be established) or question-answering dialogs (which fill out ellipsed answers using expectations about the answers to a question). These expectations are also encoded as requests but the source of the request comes from the understanding system itself. The first and second of these techniques implement word meaning selection based on surface semantics, since expecta- tions are set up by the requests associated with input words. The last method relies on the integration of the analyzer wirh various kinds of memory modules So that context CM assist the parser. Note that the advantages of integration are two-way. Re- course to context will often be decisive in eliminating ambiguity, thus reducing drastically the number of ambiguous readings to be considered by the analyzer. The analyzer in turn CM infon the contextual knowledge sources that their predictions have been substantiated. As a result, the absorption of the concepts APE: APE: APE: APE: APE: APE: 502 ,EEE TRANSACTlONS ON PATTERN ANAtYSIS AND MACHlNE INTELLIGENCE, VOL. PAM,.6. NO. 4, ,“LY 10~ *object (‘conrel’ type (*cause*) precon (*act’ type (nil) actor (*pp* type (#ingobj) ingtype (*med*)) postco” (*state* var (*health* part) (*PP* pati (*PP* pptok pph**O type (#person) persname (david) gender (*mast*)) type (#hod-prt) stype (*stomach*)) toward (*-5))))) ;The meaning structure for the sentence is based upon ;a” MTRANS, i.e., a mental transfer of information, ;irom David’s long-term memory (ltm) ;The information transferred is that some unknown act ;lnvolving aspirn has caused the physical state of David’s ;stomach to decline. Note that PP-memory and APE have ;disambiguated “his,” which at this point can only be ;David.” pphum0 is the memory pointer to “David,” ;David’s stomach” is pptokl, and “aspirin” is pptok0. ;DSAM starts on the second sentence APE: sent = (he perf$ take it period) APE: current word is “he” APE: requesting referent: singular, masculine PPMEM: possible referents: (pphum0) C-List = (apc35) apc35: (*pp’ pptok pphum0 type (#person) persname (david) gender (*ma&)) ;APE gets “David” as referent of “he.” current word is “take” C-list = (apc35 apc37) apc37: (*vel* vl (*act* actor (nil) course (nil) type @course)) v2 (*act* actor (nil) object (nil) type (*ingest*)) ;take is either Scourse or *ingest* request: subject, person from %course found: apc35 action: fti actor of acourse request: subject, person from *ingest* found: ~apc35 action: fti actor of *ingest* ;both sense of “take” accept David as actor. current word is “it” requesting referent singular, neuter PPMEM: possible referents (pptok0 pptokl) APE: C-list = (apc37 apc5 1) apc51: (*vel* v0 (‘pp* pptok pptok0 type (figobj) ingtype (*mad*)) C”L:,INGPORD AND PAZZANI: WORD-MEANING SELECTlON APE: APE: v0 (*pp. pptok pptokl type (#hod-prt) stype (*stomach*) part (*pp. pptok pphum0 type (#person) persname (david) gender (*mast*))) ;the VEL of possible referents of “it” request: object, *course* from Scourse ; the object of take can be a course; not found request: object, drug from *ingest* ; the object of take can be a drug found: pptok0 In VEL apd 1 ; the “drug” referent of “it” was found ACTION: fa object of *ingest* APE: assert VEL apd 1 is pptok0 ; selectional restrictions enable the ; compressing of the VEL formed from ; the reference of “it”. APE: assert VEL apc37 is apc42 ; the *ingest* sense of “take” apc42 ; is asserted compressing the VEL ; because l l”gest* expectation to find ; an object was met, and Scourse ; could not fmd an object APE: sentence concept: apc42 ; The fmal representation of the sentence: apc42: (*act* object (*pp. pptok pptok0 type (#mgobj) ingtype (*med*)) mode (*tf*) actor (*pp. pptok pphum0 type (#person) persname (david) gender (*mast*)) type (*Ingest*)) In this example, “it” has two possible antecedents. Selec- tional restrictions imposed bi one sense of “take” choose.& intended referent. This also disarnbiguates “take.” F. Word-Meaning Selection in Scriptal Context Throughout this paper, we have argued that understanding must be done in context. Contextual information is gleaned from “knowledge structures” which encode people’s repeated experiences in familiar (i.e., scriptal) and not-so-familiar (i.e., planning) situations. Simulating such knowledge structures for the machine gives it a source of “experience” by which it Can evaluate new input. Situational scripts are the model of context used by DSAM. A script consists of a set of roles, the standard participants in the script; a set of possible entry conditions which describe the state of the roles at the beginning of the episode; a szt of scenes, Containing the events which typically occur in a script; the Qusal and temporal relationships among the events; and a set of possible resulting states of the script. Ail these items are expressed in language-independent conceptual dependency patterns. The expert process called the script applier attempts to locate the conceptualization produced by the analyzer in its collection of scripts. If it succeeds in this, it CM build a” Inference chain of events, both those which were explicitly mentioned and those which can be reasonably assumed to have occurred, as well. This “trace” through a script is the story representation which the machine consults to demonstrate its “understanding,” e.g., by summarization or question.answering. The understanding process depends critically on the system’s ability to select word meanings. In DSAM, the event patterns of the script have been augumented so that each event can haves a named request associated with it. Such requests enable the script applier to inform APE of its expectations in a format APE can use. This gives APE the common sense knowledge we share of the domain being currently considered. An expectation-based system such as DSAM can have the problem of combinatorial explosion unless there is some method of controlling the number of expectations. DSAM 504 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. VOL. PAM,-% NO. 4. JULY I’ 89 contains a windowing mechrrnism (described in [7]), which keeps track of what has been seen and what is immediately expected, as the story follows a path through a script. This mechanism is aided by the communication channel between the analyzer and the script applier. In addition to sending concepts as soon as they tie completed, APEInforms the script applier of the relations between concepts specified by words such as “as; “after,” and “because.” With knowledge of the relation between a concept just seen and the one currently being formed, the script applier can give the analyzer the named requests for predicted events. As a” example of this communication consider the simple story: (14) As David left the restaurant, he left a tip. The second use of “left” must be disambignated by context, i.e., by the restaurant script. The Appendix contains a” an- notated protocol of DSAM performing this operatidn. Here we sketch the main features of the understanding process. When APE has completed analyzing the first clause of (14), it places its result where the memory modules @‘P-memory and the script applier) can see it. The script applier finds a match for this In the “leaving scene” of the restaurant script. At this point, since “as” Indicates that the next concept to be produced wilI be in the causal/temporal vicinity of “leaving,” the script applier can form some fairly specific predictions. It expects, among other things, that the customer will give a tip to the waiter. This expectation embodies a rule which people have about restaurants. The script applier thus sends a named request for the tipping event (and others as well) to APE, which looks it up and activates it. The request has two pur. poses. First, if the predicted concept is found in the input, APE can tell the script applier that its expectation has been substantiated, thus eliminating the need for memory search. Secondly, if APE has a conceptualization which is ambiguous, but one reading is expected by context, i.e., by the named re. quest, that one will be asserted as the proper reading. IV. SOME CONCLUSIONS We have approached the word meaning selection problem from a semantic point of view. This process is proposed as one which unities the problems of word-sense disambiguation, defmite noun-phrase (Including pronominal) reference, and discourse ellipsis, phenomena which are normally considered separately. We identified surface semantics, general world knowledge and episodic or discourse context as three sources of expectations which aid In this process. We then described a computer algorithm, the word-meaning selector, capable of using all three. The algorithm is part of a working conceptual analyzer, APE, which is in turn part of two different computer “iinderstanding” systems. Although these systems are typical thy artificial intelligence programs, they work weU enough for us to believe that “real” text-processing systems capable of flexible and reasonably deep (although not real-time) compre. hension could be designed using them as models. Our approach does have several limitations. We have pur- posely ignored any notion of favored meaningsorprobabilities. With some difficulty, for example, we could compute that the word “ring” refers to a piece of jewelry 85 percent of the time. When all else failed, we could use this sense. This Would work well (85 percent of the time, in fact), but we do not believe it has a place in a cognitive theory. We have also ignored syntactic phenomena, feeling that’ “conceptual” factors are more fundamental. Many words have multiple parts of speech (i.e., the senses belong to differ. ent parts of speech). Syntactic knowledge would help to eliminate senses which belong to a certain part of speech. Fur example, in: Hearing aids the blind. a syntactic analysis would favor the verb sense of “aid,” which would help In the conceptual analysis of this sentence. Just as the control structure of ELI limited its power, the use of requests in APE creates some problems. First of all, requests tend to make word definitions long and awkward, Secondly, requests either fire or they do not. This limits the ability of the rules to use surface semantics. The tests of :e. quests which ffl slots tend to look for only very general re- mantic categories to account for all possible cases. This limits the power of selectional restrictions when it is necessary to choose among components of a VEL which are similar. A fmal problem with requests In the VEL format is that a mean- ing selection decision, once made, is Irrevocable. APE cur- rently has no capability to “back out” of a decision of this sort. A notion of minimal requirements and more specific optional restrictions which increase the certainty of the selec- tion would be useful. A type of ambiguity we have not considered is caused by modifiers. For example, in “small car salesman,” “small” could refer to the car or the salesman. The problem here for the analyzer is not in disambiguating a VEL, which the current process should be able to do, but in creating one in the first place. This is because the request associated with “small” that looks for a concept to modify is satisfied when it fmds one. A more general approach would be to look for all such concepts, and create a VEL if there is more than one. Finally, the use of named requests to make high-level pz- dictions available to the analyzeer is unwieldy and difficult to generalize beyond simple scriptsl or planning contexts. What is really needed is a separate “expectation expert,” which would match expectation patterns from whatever source against the stream of concepts flowing In from the outside world, or circulating internally. How such a” expert would be designed is only very dimly understood at present. Nevertheless, in spite of these shortcomings, we believe that the cooperative word-meaning selector is a viable approach to a key problem in applying knowledge to understand natu:al language. Therefore, it provides a first-pass model of one type of processing the machine intelligences of the future will have to perform. IMPLEMENTATION NOTE The implementations of DSAM and ACE discussed in this paper run on a PDP-11/60 minicomputer under the UNIX operating system [29]. Both understanders are configured *s C”L’.:NGFORD AND PAZZANI: WORD-MEANING SE,XCT,ON APPLY: backbone match on rs4 with bindings ((&rest . apcl4) (&cust apc4)) APPLY: need bindings: ((&cust . apc4)) ppmem: requesting (setoutf ppmem ((&cust . pphum0))) ;ppmem says that david could be the customer ;it has been established that pptok0 (apcl4) “the ;rest*ur*nt*’ is the rest*urant APPLY: instantiated event rs4 APPLY: context active: Srestaurant ;rs4 has been established APPLY: hi-level predictions: (exp-tip) ;at the time of leaving, a tip is expected APPLY: new role bindings: (cust pphum0) (&rest pptok0)) ;meanwhile, back in APE APE: request object, info for *mtrans* FOUND: apc48 in VEL apc43 ACTION: fdl object of *mtrans* APE: assert VEL ap43 is apc48 “‘tip” is info APE: ’ assert VEL apc30 is apc40 ;le*vS is *mtrans* APE: request: object, location for *ptrans* not found APE: request: object, money for l atrans* FOUND: apc46 in VEL apc43 ACTION: fill object of *atrans* APE: assert VEL ap43 is apc48 ‘“tip” is money APE: ’ assert VEL apc30 is apc34 ;“le*ve” is *atrans* APE: receiving named-requests (exp-tip) ;exp-tip is a named request, sent to the parser from APPLY ;*pe retrieves this request, activates it ;and considers it just like other requests ;the test looks for a” event which could be the tip ;and the action puts a confutation marker on concept ;which is used by the script applier so it does not have ;to search all the scenes for this event. APE: assert VEL apc30 is apd I ;a conflict results because of two asserts associated ;with leave. The conflict resolution scheme results in ;a new vel (apdl) with a” atrans and a mtrans component. ;note that “tip’s” disambiguation is not complete ;however, its disambiguation is now dependent on ;the disambiguation of “leave” APE: C-list = (*pcO apd 1) (*vel* v0 (*act* *object (*Info* ref (‘indef’) type (nil)) actor (*pp* pptok pphum0 type (#person) persname (david) gender (*mast*)) type (*mtrans*)) v0 (*act* object (*pp. ref (%def*) type (honey)) 508 JEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHlNE INTELLIGENCE, VOL. PAMI-6, NO. 4, JULY 1084 actor (*pp. pptok pphum0 type (#person) persn?e (david) gender (*masc.)) type (*at*****))) ;the vel for “leave a tip” APE: named request exp-tip considered APE: assert VEL apd 1 is apc34 ;one effect of this request disambigutes apcS1 ;in a” ambiguous situation, the more expected reading is preferred APE: c-list - apco APE: sentence concept: apdl *apco: (*conrel* type (*when*) cona (*act* object (*pp’ type (#person) persname (david) gender (*ma&)) actor (*pp* type (#person) persname (david) gender (*mast*)) type (*ptrans*) from (*inside* part (*pp. ref (*def*) type (#strut) *type (*restaurant*)))) conb (*act* to (*pp. type (#person) occ (*service*)) object (*pp* ref (*indef*) actor (*pp. pptok pphum0 type (#person) persname (david) gender (*mast*)) type (*arm***))) ;the final conceptualization ACKNOWLEDGMENT Igl The authors wish to acknowledge several fruitful conversa- 191 tions with L. Birnbaum and M. Selfridge, the designers of CA, concerning conceptual analysis; J. Mllstein, J. Staley, and ~101 M. Bienkowski collaborated extensively In the implementation of ACE, the academic counselor. [lIl REFERENCES 111 121 [31 141 151 161 171 L. Birnbaum and M. Setfridge, “Problems in conceptual analysis of natural language,” unpublished Dep. Comput. Sci. Res. Rep. 168, Yale Univ., New Haven, CT, 1979. D. G. Bobrow et al., ‘GUS: A frunedriven dialog system: Arrificbzl Intell., vol. 8, no. 1, 1977. 1. S. Brown and R. R. Burton, “Multiple reprcsentationsofknowl- edge for tutorial reasoning,” in Representation and Understand- ing. D. Bobrow and A. Collins, Eda. New York: Academic, 1975. LIZI [I31 I141 I. C. CarbonelI, “Subjective understanding: Computer models of [IS] belief systems,” Ph.D. dissertation, Dep. Comput. Sci., Yale Univ., New Haven, CT, Res. Rep. 150, 1979. E. Charnisk, ‘Towards P model of children’s story comprehen- I161 sion,” Ph.D. dissertation, Massachusetts Inst. Tcchnol., Cambridge, MA, Rep. Al-TR 266,1972. N. Chomsky, A~peers of the Theory of Synrx. Cambridge, MA: (171 M.I.T. Press, 1965. R. E. Cullingford, “Script application: Computer understanding of newspaper stories,” Ph.D. dissertation, Dep. Camput. Sci., Yale Univ., New Haven, CT. Res. Rep. 116, 1978. [I81 -, “Pattern-matching and inference in story understanding,” DiscourseProcesres, vol. 2, pp. 319-334, Oct. 1979. R. E. Cullingford and M. W. Krueger, “Automated explanationr zs a component of a CAD system,” in Pmt. Int. Conf. Cybem. Sot., Cambridge, MA, 1980. R. E. Cullingford, “Integrating knowledge sources for computer ‘understanding’ tasks,” IEEE Trans. Sysf., Mm, Cybern., vol SMC-11,~~. 52-60,Jm. 1981. R. Cullingford, M. Krueger, M. Selfridee. and M. Bunkowski, “Automated explanations BS e cornpan% of P CAD system: IEEE Trans. Syst., Man, Cybem., vol. SMC-12, pp. 168-18,. Mar./Am. 1982. R. &l&gfard et o/., ‘Purpasive convermtian with ACE, on academic counseling experiment,” in Pmt. IEEE hf. Con! Cybern. Sot., Seattle, WA, Oct. 1982, pp. 444-448. G. F. DeJong, “Skimming stories in real time: An experiment in integrated understanding,” Ph.D. dissertation, Dep. Camput. Sci., Yale Univ., New Haven, CT, Res. Rep. 158, 1979. 1. M. Ginmare. “Natural laneuaee orocesrinn in an .utomstiC . _, ~~~~“~~~~ r ~~~ ~ _.. _..~~ Programming domain,” Ph.D. dissertation, Dep. Camput. Sci., Stanford Univ., Stanford, CA, Res. Rep. 78.671, 1978. N. Goldman, ‘%nceptual generation,” in Concepruol Inform rion Processing. R. Shank. Ed. Amsterdun: North-Holland5 1975. G. Hkst and E. &erniak, ‘Word sense md one slot dimmbig~ tion,” in Proc. 1982 AAAI Co& Pittsburgh, PA, 1982, PP.’ 95-98. J. R. Hobbs, “Pronoun resolution,” unpublished, Dep. CamPut. Sci., City Univ. New York, New York, NY, Res. Rep. 76-L 1976.~ -, “Coherence and corcference,” Co&rive Psycbol., vol. 3. no. 1, 1979. c,%LINGPORD AND PAZZANI: WORD-MEANING SELECTION [I91 [ZOI 1211 I221 1231 1241 (251 I261 1271 I281 1291 , (301 1311 1321 1331 p4j I351 1361 I. 1. Katz and J. A. Fodor. “Tbc structure of semantic theory? Lnngua8e, vol. 39, pp. 170-210, 1963. R. L Kiiby, ‘?ILlSP for PDP-11s with memory msnagement: Dep. Comput. Sci., Univ. Maryland, College Park, l&s. Rep. TR- 546.1977. I. McDermott and C. Forgy, “Production system conflict strate- gies,” in Partem Directed Infwence Sysremr, D. Waterman and F. Hayes-Roth, Eds. New York: Academic, 1978. J. R. Mechan, ‘The Mctanovcl: ‘Writing stories by computer,” Ph.D. dissertation. Dep. Comput. Sci., Yale Univ., New Haven, CT, RN. Rep. 74,1976. B. L. Nash-Webber, “Syntax beyond the sentence: Anaphora,” in The~rericnl Issues in Reading Comprehension, G. Spiro, B. Bruce. and B. Brewer, Eds. Hillsdale, NJ: Lawrence Erlbaum, 1978. A. Newell and H. A. Simon, Hu?mn Problem Solving. Engle- wood Cliffs, NJ: Prentice-Hall, 1972. M. Marcus, A Theory of Syntactic Recognirion for Naturol Language. Cambridge. MA: M.I.T. Press, 1980. C. Rieger, “An organization of knowledge for problem solving snd comprehension,“A~i~c~llnte/l.. vol. 7, no. 2,1976. C. K. Riesbeck, ‘Conceptual &ySis: in Conceptual Informe- rion Processing, R. Shank, Ed. Amsterdam: North-HoUand, 1975. -, ‘<An expectstion-driven production system for natural Ian- 8uage understanding,” in Pattern Directed Inference SYsremS, D. Waterman and F. Hayes-Roth, Ed% New York: Academic, 1978. D. M. Ritchic and K. Thompoo”, “The UNM tirne.sharin% syr tern,” Common. Ass. Compu!. Moth., July 1974. D. RumeLhart and D. Norman, “‘Dte active structunl network,” in Explorafionr in Cognition, D. Norman, D. Rumelhart, and LNR Research Group_,~_S+ ~q$_~?,~X:,Fr~~n, 1975. R. Schank, “Conceptual dependency: A theory of natural lan- guage understanding,” Cognitive Psychol., vol. 3, pp. 552-631. ,972~ _. ._. R. Schank, Ed., ConcepruaI Information Processing. Amster- dam: NortbJloUand. 1975. R. Schank and R. P. Abelson, Scripts, Pkms, Go&and Under. sanding. Hi&Je, NJ: Erlblbaum, 1977. R. Schank, “Predictive undersfanding,” in Recent Advances in rhe Psychology of Language, B. Campbell and 8. Smith, Eds. New York: Plenum, 1978. C. Schmidt er al., “Recognizing plans and summariring actions,” in Proc. Conf. AI and the Simu~tion of Behavior, Univ. Edin- burgh, Edinburgh, UK, 1976. S. Small, “Word expert parsing: A theory of distributed, word- based natural language understanding,” Ph.D. dissertation, Dep. Comput. Sci., Univ. Maryland, College Park, Tech, Rep. 954, 1 wu-. __ (371 A. M. Turing. “Computing machinery and intelligence,” Mind, vol. 59, pp. 433-460, Oct. 1950. (381 R. Widensky, “Understanding goal-based starits,” Ph.D. Dep. Comput. Sci., Yale Univ., New Haven, CX, Res. Rep. 140, 1978. 509 I391 Y. Wilks, “A preferential, pattern+eeklng semantinfor natural language understanding,” Anifictil Intell.. vol. 6, pp. 53-74, ,o,c A, ,“. 1401 T. Winogtad, Underrtmzding Norural Language. New York: Academic, 1972. (411’ W. A. Woods, ‘Transition network gammarsfor naturallang%@ analysis,“Commun. Ass. Compur. Mach..vol. 13, no. 10, 1970. ,, Rich& B. CuU;n~fi*d rS’66-M78) was born _., 1946. He received *‘-e B.E. (E.Ed.) de~ec from Manhattan Col- gc, Bronx, NY, in 1968, the M.S.E.E. degree cnn the Polytechnic lnstimte of Btooklyn, .n”k,vn NY_ :_ e University, New Haven, CT, in 1977. _ , ..- ..__ _ . . . of Bell Laboratories, Holmdel, jomed the faculty of the Department rical Engineering and Computer Science, _...._. -_, _. __........ ~~., Storm, in ,977. His resezch interests are in the problemaolving and natural-language processing areas of uti- ficial intelliwncs. Dr. Cull&ford is a member of the Association for Computing Ma- chinery and the Cognitive Science Society. ,,,,,,,,, ,,, ~,: ,; Michael J. Paunni was born in New York City, NY, in 1958. He received the B.S. and M.S. degen in computer science from the Univtr- sity of Connecticut, Storrs, in 1980. He is presently pursuing the Ph.D. degree at the University of California, Los Angeles. From 1980 to 1984 he was a member of the technical staff and a Group Leader of the Arti- ticial lntelligenee Technology Group at the Mitre Corporation, Bedford, MA, where he performed research in natural language understanding and Pbnrdw Mr. Puzani is a member of the Auociatian for ComputingMachinery, the ACI, and the AAAI.