¡Descarga Signal Detection Theory: Applications, Procedures, and Limitations y más Apuntes en PDF de Psicología solo en Docsity!
Psychological Bulletin 1974, Vol. 81, No. 12, 945-
SIGNAL DETECTION THEORY:
CONSIDERATIONS FOR GENERAL APPLICATION
R. E. PASTORE^1 AND C. J. SCHEIRER State University of New York at- Binghamton
While there exist a number of papers describing the theory of signal detection, it appears that many psychologists are not aware of the ease with which signal detection theory can be applied, the range of applications possible, or the limitations of signal detection theory. This paper briefly summarizes the assumptions of signal detection theory and describes the procedures, the limi- tations, and practical considerations relevant to its application. A worked example of an application of signal detection theory to the study of cognitive processes is included.
In recent years, researchers in many di- verse areas of psychology have begun to employ the theory of signal detection to sepa- rate the ability of the subject to differentiate between classes of events from motivational effects or response biases. In addition to its extensive application in sensory psychophys- ics, signal detection theory has found applica- tion in such diverse areas as speech percep- tion (Egan & Clarke, 1956), memory (Banks, 1970; Bernbach, 1967; Parks, 1966), animal learning (Rilling & McDiarmid, 1965; Su- boski, 1967), audiology (Campbell & Moulin, 1968), attention (Moray, 1970; Sorkin, Pas- tore, & Pohlmann, 1972), clinical psychology (Sutton, 1972), and sensory-evoked poten- tials (Hillyard, Squires, Bauer, & Lindsay, 1971). The purpose of this article is to re- view and briefly summarize the more common models 2 of signal detection theory, describe the procedures required to apply each model, and discuss the limitations inherent in each. While there are a number of excellent theo- retical papers describing signal detection theory and its various models (e.g., Egan & Clarke, 1966; Green & Swets, 1966; Lick-
(^1) The authors wish to thank Crawford Clark, Don Ronken, John Swets, Douglas Creelman, William Lutz, and Charlotte MacLatchy for their helpful criticisms of earlier drafts of this paper. Requests for reprints may be sent to either author, Department of Psychology, State University of New York, Binghamton, New York 13901. 2 The term model is used in this article to connote a special case of the theory based on certain clearly defined assumptions and, therefore, is limited in scope.
lider, 1959; Peterson, Birdsall, & Fox, 1954; Swets, Tanner, & Birdsall, 1961; Van Meter & Middleton, 1954), there is a clear need for a concise, unified explanation of how and when to use the various models of signal detection theory. This article attempts to ful- fill that need. The first section of this article presents a brief summary of the models of sig- nal detection theory on a general level. The second section presents practical considera- tions for the application of signal detection theory and the specific procedures used in these applications. The third section outlines the potential use of signal detection theory in several experimental situations and presents a worked example of an application to the study of cognitive processes.
OVERVIEW The purpose of this section is to provide a brief overview and summary of the theoreti- cal underpinnings of signal detection theory. For a more complete introduction and theo- retical presentation, the reader should refer to the articles by Egan and Clarke (1966), by Swets et al. (1961), or others. Signal detec- tion theory is an adaptation of statistical decision theory (e.g., Wald, 1950). A major aspect of both signal detection theory and sta- tistical decision theory concerns the specifi- cation of a set of ideal processes or observers as a standard against which a subject's per- formance is compared. While this comparison is an important aspect of signal detection theory, the specification of an ideal observer depends on the exact area or modality under 945
(^946) R. E. PASTORE AND C. J. SCHEIRER
study and the assumed capabilities of the observer. Therefore, this aspect of signal detection theory is not considered in this article; for a general discussion of the use of ideal observers, see Tanner (1961) or Tan- ner and Sorkin (1972).
General Case Signal detection theory is applicable to those situations in which two classes of events are to be discriminated. It also can be gen- eralized to situations involving more than two classes of events (Tanner, 1956; but also see Luce, 1963), although this generalization is not discussed in this article. The basic as- sumption of signal detection theory is that each decision made by the subject is based on a statistic that is derived from the many (i.e., M) characteristics of the event in question. This statistic reflects the relative probability that the observed characteristics of the event arose from one of a specific class of events. The optimum statistic for such a decision is the likelihood ratio or some monotonic trans- form of the likelihood ratio (Green & Swets, 1966). The likelihood ratio, X(«), is the rela- tive likelihood that the event, u,. arose from one as opposed to the other class of events. That is, /»(«), the likelihood that the given M-dimensional observation u arose from class i, is the product of the probability that each of the M observed characteristics arose from a class i event, and
X(«) =/!(«)//,(«). [1] The theory assumes that the subject computes (u), or some monotonic transform of A(w)[e.g., '(u) = log A.(w)], for each event and makes a response decision based on that computed value. It is further assumed that the subject adopts a fixed criterion value of A(«), called /3 and that the decision corre- sponding to any event, u, is simply a state- ment of whether (u) is greater than /?. The capability of the subject to discriminate be- tween the two classes of events is inversely proportional to the total area common to the two conditional probability density functions [/{(«); i = 1, 2]. This common area is as-
sumed to be invariant during the measure- ment interval.^3
Assumption of Normality One specific set of signal detection theory models, the Gaussian models, assumes that the two conditional probability density functions [/»(«)] are Gaussian (normal). One theo- retical basis for this assumption is that there are a large number (i.e., M) of independent characteristics of the event sampled on each observation, and that the system performs a logarithmic transform of the computed likeli- hood ratio (Egan & Clarke, 1966). It should be noted that logarithmic transforms are common in perceptual and behavioral data (e.g., Fechner's and Stevens' laws). Because of this hypothetical logarithmic transforma- tion, the separate likelihood statistics are the sums of a large number of independent fac- tors. Then, according to the central limit the- orem, the distribution of the likelihood sta- tistics approximates a normal distribution. On a practical level, the important question is whether there exists evidence that in the given experimental situation, the assumption of normality is tenable. Such evidence might be in the form of reliable results published in the literature or in a test of the assumption by the experimenter (see following sections entitled "Assumptions of Normality and Equal Variance" and "Rating Procedure"). If the Gaussian assumption cannot be justi- fied, then alternative statistics should be em-
(^3) The computation of a likelihood ratio statistic assumes that the observer knows the probability distribution for each sampled characteristic condi- tional on each of the two classes of events. Obvi- ously, the discriminaWlity of the events from the two classes of events reflects the subject's knowl- edge of the actual differences between the two classes. The assumption of a decision statistic based on the likelihood ratio is simply an assumption that the subject's knowledge of the classes of events can be used in terms of the conditional probability den- sity functions for each characteristic, ,and that this probability information is combined in an efficient, systematic manner. The theory further states that any factor (i.e., learning) that changes the subject's knowledge of these differences will alter the likeli- hood statistics, and therefore the discriminability of the two classes of events.
(^948) R. E. PASTORS AND C. J. SCHEIRER
map of data points for all possible criteria at a fixed level of sensitivity. Thus, the receiver-operating-characteristic curve is also called the Isosensitivity curve. In memory tasks, it is referred to as the memory oper- ating characteristic (MOC).
APPLICATION CONSIDERATIONS
Assumptions of Normality and Equal Variance
If there is sufficient evidence in the litera- ture to warrant the assumption of equal- variance Gaussian density functions, d' and /? may be employed to describe, respectively, the subject's ability to discriminate the two classes of events and the subject's response bias, subject to the considerations described below. If there is insufficient evidence to support the assumptions of the model (that the density functions are Gaussian and of equal variance), these assumptions should be tested directly in applying the model. The most typical method for testing these assump- tions is the use of a rating procedure (cf. Egan, Schulman, & Greenberg, 1959). This procedure should be used also if the density functions are suspected of being Gaussian, but of unequal variance, and might be employed by careful researchers even when both assumptions are supported by previous research.
Rating Procedure
With the rating procedure, the subject is asked to use N different responses that reflect the subject's confidence that a Class 1 event has occurred. Typically, five to eight confi- dence ratings or responses are employed. It is assumed that the subject operates in a manner similar to that employed in binary decision tasks (yes-no or two-alternative forced choice) and adopts N — 1 criteria separating each adjacent pair of the N re- sponses rather than the single criterion em- ployed in the binary task. The results gen- erated by the use of these N — 1 criteria are plotted as N — 1 points (%, ys), where ; = 1, 2,... , N — 1, and where x} and ys are
defined as
i Xj = ^ P (response _i_ Event 2)
and
ys = Y. P (response i] Event 1). [3]
Thus Xj and y} are the values of the distribu- tion functions for Events 1 and 2 at criterion ;'. Obviously, if the assumptions of equal- variance Gaussian functions hold, the ex- pected values of d' calculated for each point (Xj, yf) are equal and therefore not cor- related with the criterion employed by the subject. The functional relationship between y^ (the probability of a "hit") and xt (the probabil- ity of a "false alarm") describes the contour of criteria for a fixed set of density functions under a specific experimental condition. This function of equal sensitivity is the receiver- operating-characteristic curve described ear- lier. If the probabilities *» and yt are trans- formed to the equivalent z scores [x'j = z of (Xj — .5) and y'} = z of (y} — .5)], the nor- malized receiver-operating-characteristic curve can be used to test the validity of the assump- tions of normality and equal variance. If the underlying density functions are Gaussian, the normalized receiver-operating-character- istic curve will describe a linear function:
= ax' + c. (^) [4] The slope of this function, a, is equal to the ratio of the standard deviations of the two density functions (a = o- 2 /<ri), and the inter- cept, c, is related to the distance between the distribution means, If the receiver-operating-characteristic curve exhibits a systematic deviation from linearity, the Gaussian assumption may be invalid. If this deviation from linearity is large, but not systematic, there exists an actual deviation from normality and/or a large error factor that may be correlated with the criterion of the subject. Any criterion-correlated error factor will distort the form of the normalized receiver-operating-characteristic curve. How-
SIGNAL DETECTION THEORY 949
ever, an equally important concern is that any large error factor might mask an actual deviation from linearity. Such error factors may be due to a number of different prob- lems, including a high degree of criterion variability and/or an insufficient number of trials employed in the experiment (see sec- tion entitled "Number of Trials"). If the error factor is large, or is suspected of being large, the interpretation of the results should reflect this fact. The linear function best describing the data, expressed as standard normal deviates, should be estimated by standard curve-fitting operations.^6 If the data are adequately de- scribed by a linear function, the Gaussian assumption is supported. It should be noted, however, that this use of cumulative rating data to generate the receiver-operating-char- acteristic curve provides data points that are not independent and, at a minimum, imposes a monotonic relationship between successive data points. If the Gaussian assumption is not rejected, the equal-variance assumption is tested with the slope, a, of the normalized receiver- operating-characteristic curve (Equation 4). If the slope is approximately equal to 1.0, the equal-variance assumption is not rejected and the equal-variance Gaussian model may be legitimately employed with d' and ft as pa- rameters. While the values of d' estimated from the N — 1 criteria are not totally inde- pendent estimates, the mean or median value of d' may be employed as the estimate of ob- server sensitivity. The measures derived for
(^5) Many researchers (e.g., Swets et al., 1961) have used simple visual fits to determine the "best-fit- ting" straight line. This crude method is probably sufficient for most proposed uses of the function. Conventional least squares curve-fitting procedures are theoretically inappropriate because both variables are dependent variables and subject to error. For rough approximations, this problem is of minor importance since the error introduced is likely to be small relative to the noise in the data. However, for researchers interested in precise estimates of the parameters of receiver-operating-characteristic curves and in a test for goodness-of-fit of the theoretical model, maximum-likelihood estimators giving exact fits have been developed by Ogilvie and Creelman (1968).
the unequal-variance Gaussian model, dis- cussed in the next section, and the nonpara- metric model, discussed in the section entitled "Nonparametric Model," may be more desir- able than those derived from the equal-vari- ance model since fewer restrictive assumptions are involved.
General Gaussian Case If the equal-variance assumption is vio- lated, d' and /? will be correlated to a degree that is related to the deviation from equality of variance. The general Gaussian (or un- equal-variance Gaussian) model is applicable when the Gaussian assumption is justified, in- dependent of the relative magnitude of the variances. Application of the general Gaussian model requires knowledge of the slope, a, of the normalized receiver-operating-character- istic curve (Equation 4) which may be esti- mated with the rating procedure (see section entitled "Rating Procedure"). The basic goal of the general Gaussian model is to develop a statistic that describes sensitivity, is inde- pendent of the subject's criterion, and reflects the average spread or variance of the two distributions. Several statistics use the fact that when ft = 1.0, ft is equidistant from the two distribution means in terms of standard normal deviates (z scores) for each of the given distributions. At J3 = 1.0, the "hit rate" for the two classes of events [P( response _i_ event i)] are equal. Thus d' computed at y8 = 1.0, the minimum total error criterion (given equal probability of presentation for the two classes of events), will be based on the average standard deviation with equal weighting given to the two distributions. This minimum error criterion is the negative diago- nal (y' = — x') of the receiver-operating- characteristic space (see Figure 1). The co- ordinates (x'm, y'm) of the intersection of the estimated receiver-operating-characteristic curve (Equation 4) and the negative diagonal define the value of d' for this minimum error criterion. The value of d' for this point is equal to the distance along the negative diag- onal from the positive diagonal (chance line) scaled in terms of the difference between the coordinates (y'm — x'm), and is called d's
SIGNAL DETECTION THEORY 951
the formula:
A. = (1/2) Z (*y+i - *y) (yi+1 + yy), [7]
where Xj and y^ are defined by Equation 3. Pollack and Hsieh (1969) have used Monte Carlo methods to sample from various den- sity functions in order to investigate the sampling distributions of Ag and d'e (dis- cussed in a previous section entitled "General Gaussian Case"). They determined that when the normality assumptions of signal detection theory are satisfied, d'e and a Gaussian trans- form of Ag, N(Ag),w& related by the formula
N(Ag) = [.707 - .2341og 2 [8]
They found that the empirically determined values of N(Ag) tended to overestimate the values of d'e by l%-6%. Hodos (1970) developed a nonparametric measure of criterion or bias. This measure was based on the fact that the negative diag- onal of the unit square represents the locus of points where the subject would be equally likely to respond "i" or "j" given ambiguous stimulus conditions. The measure reflects the degree to which a data point deviates from the negative diagonal relative to the maxi- mum possible deviation. A computational formula for the nonparametric measure of criterion, ft', based on the Hodos measure, has been developed by Grier (1971). The formula is:
(? = 1 - *,-(! - *,)/?,-(! - *), [9]
where Xi and yt are defined in Equation 3.
Criterion Stability If the criterion adopted by the subject is not stable during any given session, the vari- ability of the criterion will affect the results. The presence of criterion variability cannot be detected easily, and will have the same effect on the results as an increase in the variance of both likelihood density functions. Criterion variability therefore decreases the estimate of d' by an amount that is related to the size of the criterion variance without
actually affecting the true discriminability of the two classes of events. Obviously, any experimental manipulation that affects cri- terion variability will alter the estimate of d'. Safeguards against criterion variability in- clude the use of trained subjects, strict instructions to the subjects about maintaining a stable criterion, and strict definitions of the subject's response classes. Since the subject's criterion is partially determined by the expec- tation of the probability of presentation of the two classes of events, the subject should be made aware of the absence of sequential dependencies across trials. While it may be reasonable to assume that the criterion employed by a single subject during any measurement session (block of trials) is stable, it is less reasonable to assume that the subject will employ the same cri- terion across sessions, or even across separate blocks of trials within a session. Therefore, only the data for a single block of trials should be used to estimate a value of d'. The estimates of d' from the various blocks of trials may then be averaged.
Malingering The positive diagonal of the receiver-oper- ating-characteristic space (x' = y') defines chance performance. Under the equal-variance Gaussian model, the receiver-operating-char- acteristic curve that corresponds to the posi- tive diagonal is generated under the condition of exact equality for the two density functions. Data points below this chance line can be generated only by (a) measurement error or (b) the subject performing the discrimination and then emitting a response that is incon- sistent with the computed decision statistic [A'(y)l- If a subject consistently produces data that fall below the chance line, there is justification to assume that the subject can perform the discrimination, but is malingering.
Number of Trials In applying signal detection theory, the ex- perimenter is assuming that there are two fixed internal probability density functions, and the subject has established a fixed cri-
(^952) R. E. PASTORE AND C. J. SCHEIRER
terion along the dimension (decision axis) on which these functions lie. The purpose of the experiment is to estimate the area [P (re- sponse _"i"_ event j)] in the tail of each of the two distributions from the relative fre- quencies of the responses. The expected standard error in estimating the probablity [P("i"\j)] as a function of both tie sample size and the expected value of this probability is p • q/s, where p is the expected value of the probability, q = 1 —p, and s is the sample size. The expected error of estimation for d' can be obtained by applying a z transforma- tion to p — (p • q/s). Green and Moses (1966) found that the actual error involved in estimating this parameter is slightly larger than predicted by this assumption of binomial variability. Pollack and Hsieh (1969), in a computer simulation, found that the error variance in Ag was slightly smaller than pre- dicted by the assumption of binomial vari- ability. Therefore, the expected binomial vari- ability would seem to reflect the magnitude of error to be expected in a given measure. Obviously, the use of only a small number of trials for one or both of the event classes re- sults in a large expected error in estimates of the parameters of the model. Furthermore, reliable estimates of sensitivity require a large number of trials when based upon extreme values of P("i"\j).
APPLICATIONS OF SIGNAL DETECTION THEORY
Since signal detection theory provides the researcher with a means of evaluating inde- pendently both the ability of an organism to discriminate between classes of events and motivational or other response effects, it can be a powerful research tool having applica- tion in a variety of different experimental settings. It is the purpose of this section to outline some potential applications of signal detection theory in areas of psychology where this method has not been widely used. The applications discussed include the evaluation of (a) the state of the organism or environ- ment, (b) the relationship between stimuli and potential or actual responses, and (c) the independence of "channels" for processing stimulus information. Finally, a more tradi-
tional example from memory work is pre- sented in some detail to provide a worked example for persons unfamiliar with the com- putational procedures involved in a signal detection theory analysis.
Evaluating the Condition of Subjects or Environment
The ability of a subject to perform detec- tion, discrimination, or recognition tasks can be altered by a number of conditions includ- ing the psychological or physiological condi- tion of the subject (e.g., behavioral or organic dysfunction), the existence of a drug state, or the imposition of an external stimulus. In many cases, however, it is unclear whether the performance difference is due to changes in the ability of the subject to perform the task or changes in the response tendencies of the subject. Signal detection theory may be used in a between-groups design to evaluate the cause of the observed differences between an altered and a control population. Simi- larly, signal detection theory could be used in a pretest-posttest design to investigate the locus of performance differences as a result of pharmacological or surgical interventions. A somewhat less obvious potential applica- tion occurs in the area of motivation. If an experimenter discovers that rats initially ex- hibit a preference for a given solution over water, but after two months of continuous ad libium intake of the solution exhibit no differential preference, the experimenter does not know whether the motivation of the tats or their ability to discriminate between the two solutions (^) ;has been altered. However, by using the given solution and water as the discriminative stimuli with either an appeti- tive or avoidance conditioning technique, the researcher could apply nonparametric mea- sures of signal detection theory to the per- formance data to evaluate the nature of this change.
Evaluating Response Factors Performance in any given task is deter- mined by two main classes of variables: those -that affect the discriminability of
(^954) R. E, PASTORS AND C. J. SCHEIRER
lus; signal detection theory measures of sensi- tivity were computed for each of these two stimulus channels. A comparison of the signal detection theory measure under the simple and simultaneous condition provides an indi- cation of the degree of interaction between the two tasks, which may be evaluated in quantitative terms by methods developed by Taylor, Lindsay, and Forbes (1967). Because there is independence across stimuli and across most stimulus-response categories in the response matrix, it is possible to calculate a signal detection theory measure of sensitiv- ity in one channel conditional on the simul- taneous stimulus, response, or outcome event in the other channel. With modern data- handling techniques, partitioning of the data in this manner has become a simple matter. Pastore and Sorkin (1972) examined the ef- fects on sensitivity in a single sensory chan- nel as a function of the various possible stimulus events and outcomes in the second channel in a simultaneous two-channel detec- tion paradigm. This technique of analysis has also been successfully employed by Harvey and Treisman (1973) in a simultaneous task and by Sorkin, Pohlmann, and Gilliom (1973) in a successive two-channel task.
Evaluating Memory Processes Lutz and Scheirer (1974) investigated differences in the processes involved in the encoding of visually presented verbal and pictorial stimuli. Each subject was presented with a series of 190 stimulus items; each item was presented for either .25, .50, 1.00, or 2.00 seconds with a fixed interstimulus interval of either .25, 1.00, or 2.00 seconds. All conditions in the 4 X 3 X 2 (Presentation Interval X Interstimulus Interval X Stimulus Type)' factorial arrangement were presented to independent groups of 12 subjects. At the beginning of the session the sub- jects were told that a series of items would appear on the screen in front of them. The subjects were instructed to "pay careful at- tention to each item.. ." since the subjects would "later... be given a test based on these items." Following the presentation of the 190 stimulus items, the subjects were told
that a second series of items would be pre- sented, each for a few seconds. They were told that some of the items had been pre- sented previously and some had not, and were given the following instructions:
When an item appears, look at it and decide whether the item appeared in the first series. You should record your decision on the answer sheet. There are six categories you can respond with: +H-+ if you are definite that you have seen the item before; ++ if you believe that you have seen the item before;
- if you guess that you have seen the item before; — if you guess that you have not seen the item before; if you believe that you have not seen the item before; if you are definite that you have not seen the item before. For each stimulus you should respond with one and only one of the above categories. For each item that appears, circle the appropriate symbol on the answer, sheet as soon as you have made your decision. Try to use all six categories but only where appropri- ate ... In summary, when the first item appears look at it, decide whether the item appeared in the first series... Then wait for the next item to appear [p. 317].
AH instructions were read aloud to the sub- jects, with the category definitions typed on a card given to the subject for reference during the experiment. A series of 120 test stimuli were presented to the subject, 60 ;of these test stimuli were randomly chosen from the original 190 items. These "old" items were randomly mixed with 60 "new" items that were not in the original set.... The relative frequency of responding with each of the six categories to each of the two classes of events is shown in Table 1 for two of the subjects. These rating response data were converted to cumulative response prob- abilities as described by Equation 3 and then plotted as receiver-operating-characteristic curves. Figure 1 is the normalized receiver- operating-characteristic curve for the two sub- jects reported in Table 1. The upper and right margins of the figure are delineated in z-score units. The lower and left margins are delineated in terms of probabilities. The data points are labeled according to the limits
SIGNAL DETECTION THEORY^955
TABLE 1 RELATIVE AND CUMULATIVE FREQUENCIES WITH EACH CATEGORY TOR THE SUBJECTS DESCRIBED IN TEXT° AND IN FIGURE 1
Subject
1
Total
2
Total
response i
—
—
P(j'lnew)
. . . . . .
. . . . . .
P(»|old)
. . . . . .
. . . . . .
SP(t|new)
. . . . .
. . . . .
SP(»Iold)
. . . . .
. . . . .
»See section entitled "Rating Procedure.'
of summation indicated in Equation 3 and Table 1. The linear function describing each set of data is plotted in Figure 1. The data for the second subject appear to be curvilinear. Had the data for a majority of the other subjects been curvi- linear, the Gaussian assumption would have to be rejected. However, a small and approxi- mately equal number of receiver-operating- characteristic curves were curvilinear in each direction, and most functions were linear (e.g., see the curve for Subject 1 in Figure 1). Therefore, the Gaussian assumption was held to be supported by the data and the few deviations from linearity were assumed to be due to error. The linear functions for the two subjects plotted in Figure 1 have estimated slopes of .91 and .85, respectively. While these slopes do not differ substantially from the slope of unity required by the assumption of equal- variance Gaussian distributions, larger devia- tions were found for a number of subjects. Since acceptance of the equal-variance as- sumption is therefore tenuous and since the rating procedure allows the use of the general Gaussian model (see sections "Rating Proce- dure" and "General Gaussian Case"), d'g was used as the measure of discriminability rather
than d'. The negative diagonal (minimum error criterion) of the receiver-operating- characteristic space in Figure 1 is delineated in units of d's. The intersection of this diag- onal with the linear regression lines for the obtained data yield d's estimates of 1.94 and 1.14 for the two subjects. The corresponding values of the nonparametric area measure of sensitivity, Ae, computed with the use of Equation 7, are .892 and .785. These are estimates of the ability of the subjects to dis- criminate the two classes of events indepen- dent of the criteria employed and any dif- ferences in variability within the classes of events. Using any positive (+, ++, + + + ) response as a response indicating an "old" stimulus and any negative response (—, — — , ) as a response indicating a "new" stimulus, the probability of a correct response was computed for each subject. A within-cell product-moment correlation of .91 was ob- tained between d's and the probability of being correct. This high correlation appears to be at least partially due to the use of a strict set of criterion categories. While this procedure was intended to minimize within-subject vari- ability, it also appears to have caused most subjects to adopt a set of criteria whose
SIGNAL DETECTION THEORY^957
Banks, W. P. Signal detection theory and human memory. Psychological Bulletin, 1970, 74, 81-99. Bernbach, H. A. Decision processes in memory. Psychological Review, 1967, 74, 462-480. Campbell, R. A., & Moulin, L. K. Signal detection audiometry: An exploratory study. Journal of Speech and Hearing Research, 1968, 11, 402-410. Clarke, F. R., Birdsall, T. G., & Tanner, W. P., Jr. Two types of ROC curves and definitions of parameters. Journal of the Acoustical Society of America, 1959, 31, 629-630. Egan, J. P. Recognition memory and the operating characteristic. (Tech. Note AFCRC-TN-58-51) Indiana University: Hearing and Communication Laboratory, 19S8. Egan, J. P., & Clarke, F. R. Source and receiver behavior in the use of a criterion. Journal of the Acoustical Society of America, 1956, 28, 1267-
Egan, J. P., & Clarke, F. R. Psychophysics and signal detection. In J. B. Sidowski (Ed.), Experi- mental methods and instrumentation in psychol- ogy. New York: McGraw-Hill, 1966. Egan, J. P., Schulman, A. I., & Greenberg, G. A. Operating characteristics determined by binary decisions and by ratings. Journal of the Acoustical Society of America, 1959, 31, 768-773. Eijkman, E., & Vendrik, A. J. H. Can a sensory system be specified by its internal noise? Journal of the Acoustical Society of America, 1965, 37, 1102-1109. Emmerich, D. S., Gray, J. L., Watson, C. S., & Tanis, D. C. Response latency, confidence, and ROCs in auditory signal detection. Perception and Psychophysics, 1972, 11, 65-72. Green, D. M. General prediction relating yes-no and forced choice results. Journal of the Acoustical Society of America, 1964, 36, 1042. Green, D. M., & Moses, F. L. On the equivalence of two recognition measures of short-term memory. Psychological Bulletin, 1966, 65, 228-234. Green, D. M., & Swets, J. A. Signal detection theory and Psychophysics. New York: Wiley,
Gourevitch, V., & Galanter, E. A significance test for one parameter isosensitivity functions. Psycho- metrika, 1967, 32, 25-33. Grier, J. B. Nonparametric indexes for sensitivity and bias: Computing formulas. Psychological Bulletin, 1971, 75, 424-429. Harvey, N., & Treisman, A. M. Switching attention between the ears to monitor tones. Perception and Psychophysics, 1973, 14, 51-59. Healy, A. F., & Jones, C. Criterion shifts in recall. Psychological Bulletin, 1973, 79, 335-340. Hillyard, S. A., Squires, K. C., Bauer, J. W., & Lindsay, P. H. Evoked potential correlates of auditory signal detection. Science, 1971, 172, 1357-
Hodos, W. Nonparametric index of response bias for use in detection and recognition experiments. Psychological Bulletin, 1970, 74, 351-354. Licklider, J. C. R. Three auditory theories. In S.
Koch (Ed.) Psychology: A study of a science. Vol. 1. New York: McGraw-Hill, 1959. Luce, R. D. Detection and recognition. In R. D. Luce, R. R. Bush, & E. Galanter (Eds.), Hand- book of mathematical psychology. New York: Wiley, 1963. Lutz, W. J., & Scheirer, C. J. Encoding processes in pictures and words. Journal of Verbal Learn- ing and Verbal Behavior, 1974, 13, 316-320. Markowitz, J., & Swets, J. A. Factors affecting the slope of empirical ROC curves: Comparison of binary and rating responses. Perception and Psy- chophysics, 1967, 2, 91-100. Moray, N. Time sharing in auditory perception: Effect of stimulus duration. Journal of the Acous- tical Society of America, 1970, 47, 660-661. Murdock, B. B., Jr. The criterion problem in short- term memory. Journal of Experimental Psychol- ogy, 1966, 72, 317-324. Ogilvie, J. C., & Creelman, C. D. Maximum-likeli- hood estimation of receiver operating character- istic curve parameters. Journal of Mathematical Psychology, 1968, 5, 377-391. Parks, T. E. Signal-detectability theory of recogni- tion-memory performance. Psychological Review, 1966, 73, 44-58. Pastore, R. E., & Sorkin, R. D. Simultaneous two- channel signal detection. I. Simple binaural stim- uli. Journal of the Acoustical Society of America, 1972, 51, 544-551. Peterson, W. W., Birdsall, T. G., & Fox, W. C. The theory of signal detectability. Transactions IRE Professional Group on Information Theory, 1954, 4, 171-212. Pike, R. Response latency models for signal detec- tion. Psychological Review, 1973, 80, 53-68. Pollack, I., & Hsieh, R. Sampling variability of the area under the ROC-curve and of d'',. Psycho- logical Bulletin, 1969, 71, 161-173. Pollack, L, & Norman, D. A. A non-parametric analysis of recognition experiments. Psychonomic Science, 1964, 1, 125-126. Pollack, L, Norman, D. A., & Galanter, E. An efficient non-parametric analysis of recognition memory. Psychonomic Science, 1964, 1, 327-328. Rilling, M., & McDiarmid, C. Signal detection in fixed-ratio schedules. Science, 1965, 148, 526-527. Sorkin, R. D., Pastore, R. E., & Pohlmann, L. D. Simultaneous two-channel signal detection. II. Correlated and uncorrelated signals. Journal of the Acoustical Society of America, 1972, 51, 1960-
Suboski, M. D. Signal detection methods in the analysis of classical and instrumental discrimina- tion conditioning experiments. Proceedings of the 75th Annual Convention of the American Psycho- logical Association, 1967, 2, 37-38. Sutton, S. Fact and artifact in the psychology of schizophrenia. In M. Hammer, K. Salzinger, & S. Sutton (Eds.), Psychopathology. New York: Wiley, 1972. Swets, J. A. (Ed.). Signal detection and recogni- tion by human observers. New York: Wiley, 1964.
958 R. E. PASTORE AND C. J. SCHEIRER
Swets, J. A. The relative operating characteristic in psychology. Science, 1973, 182, 990-1000. Swets, J. A., Tanner, W. P., Jr., & Birdsall, T. G. Decision processes in perception. Psychological Review, 1961, 68, 301-340. Tanner, W. P., Jr. Theory of recognition. Journal of the Acoustical Society of America, 1956, 28, 882-888. Tanner, W. P., Jr. Physiological implications of psychophysical data. Annals of the New York ^ Academy of Sciences, 1961, 89, 7S2-76S. *
Taylor, M. M., Lindsay, P. H., & Forbes, S. M. Quantification of shared capacity processing in auditory and visual discrimination. Ada, Psycho- logica, 1967, 27, 223-229. Van Meter, D., & Middleton, D. Modern statistical approaches to reception in communication theory. Transactions IRE Professional Group on Informa- tion Theory, 1954, 4, 119-141. Wald, A. Statistical decision functions. New York: Wiley, 19SO. ' (Received January 16, 1974)
