Download Computing Effect Sizes from 'set aside' Lambda Weights in Statistical Tests: A Guide and more Schemes and Mind Maps Design in PDF only on Docsity!
BASIC EFFECT SIZE GUIDE WITH SPSS AND SAS SYNTAX
Gregory J. Meyer, Robert E. McGrath, and Robert Rosenthal Last updated January 13, 2003
Pending:
**1. Formulas for repeated measures/paired samples. (d = r / sqrt(1-r^2)
- Explanation of 'set aside' lambda weights of 0 when computing focused contrasts.
- Applications to multifactor designs.**
SECTION I: COMPUTING EFFECT SIZES FROM RAW DATA.
I-A. The Pearson Correlation as the Effect Size I-A-1: Calculating Pearson's r From a Design With a Dimensional Variable and a Dichotomous Variable (i.e., a t- Test Design). I-A-2: Calculating Pearson's r From a Design With Two Dichotomous Variables (i.e., a 2 x 2 Chi-Square Design). I-A-3: Calculating Pearson's r From a Design With a Dimensional Variable and an Ordered, Multi-Category Variable (i.e., a Oneway ANOVA Design). I-A-4: Calculating Pearson's r From a Design With One Variable That Has 3 or More Ordered Categories and One Variable That Has 2 or More Ordered Categories (i.e., an Omnibus Chi-Square Design with df > 1).
I-B. Cohen's d as the Effect Size I-B-1: Calculating Cohen's d From a Design With a Dimensional Variable and a Dichotomous Variable (i.e., a t- Test Design).
SECTION II: COMPUTING EFFECT SIZES FROM THE OUTPUT OF STATISTICAL TESTS AND TRANSLATING ONE EFFECT SIZE TO ANOTHER.
II-A. The Pearson Correlation as the Effect Size II-A-1: Pearson's r From t- Test Output Comparing Means Across Two Groups. II-A-2: Pearson's r From 2 x 2 Chi-Square Output. II-A-3. Pearson's r From F Test Output When Just Two Groups Have Been Compared. II-A-4: Pearson's r From F Test Output When More Than Two Groups Have Been Compared. II-A- 4 - a: Scenario 1 - When given group M s and F or t from a focused contrast II-A- 4 - b: Scenario 2 - When given group M s, SDs, and n s II-A- 4 - c: Scenario 3 - When given group M s and results from an omnibus F test II-A-5: Pearson's r From the Output of an Omnibus Chi-Square Test (i.e., df > 1). II-A- 5 - a: The Rosnow and Rosenthal (1996) approach II-A- 5 - b: Reconstructing the database II-A-6. Pearson's r from Cohen's d. II-A- 6 - a: The exact formula. II-A- 6 - b: The approximate formula.
II-B. Cohen's d as the Effect Size II-B-1: Cohen's d from t- Test Output. II-B- 1 - a: When the size of each group is known. II-B- 1 - b: When the size of each group is unknown but df is available.
II-B-2: Cohen's d From F Test Output When Just Two Groups Have Been Compared. II-B-3: Cohen's d from Pearson's r.
General Note: When computing r and d according to the procedures in this guide, r and d are effect size measures like those used in a meta-analysis. As such, r and d should be given a positive sign when the result is consistent with the a priori hypothesis and a negative sign when the result is in the direction opposite of that specified by the hypothesis. This point is emphasized throughout the guide and should be stated in the text of any manuscript reporting these results.
I-A-2: Calculating Pearson's r From a Design With Two Dichotomous Variables (i.e., a 2 x 2 Chi-Square Design).
In a 2x2 chi-square design there are two dichotomous variables. To compute r , each of the dichotomous variables should be coded so scores take on the values - 1 and 1 (although any two values could be used). Next, we correlate the first dichotomous variable with the second dichotomous variable. This type of correlation is often referred to as a phi coefficient but it is simply Pearson's r with both variables dichotomous. In other words, a phi coefficient is not different from a Pearson correlation.
To compute r from this kind of design using SPSS or SAS syntax, we open the dataset containing raw scores on our two dichotomous variables. For the sake of simplicity, the following syntax commands assume that these two variables have been labeled "dichot_a" and "dichot_b" and the syntax will have to be modified so it uses the correct variable names for your dataset.
The SPSS syntax is as follows. CORRELATIONS /VARIABLES= dichot_a dichot_b /PRINT=TWOTAIL SIG /MISSING=PAIRWISE.
The SAS syntax is as follows. PROC CORR; VAR dichot_a dichot_b; RUN;
Note. The sign of the correlations produced by any of the above commands is arbitrary. When the direction of the result is consistent with the a priori hypothesis, it should be reported with a positive sign. When the result turns out to be in the direction opposite of that specified by the hypothesis, it should be given a negative sign. This is easily accomplished by assigning the larger values on dichot_a and dichot_b in such a way that the hypothesized association leads to a positive correlation. For instance, if patients with T-Scores > 65 on MMPI-2 Scale 2 are expected to receive a depressive diagnosis more often than patients with T-Scores < 66, the MMPI variable could be coded as T > 65 = 1 and T < 66 = - 1, while the diagnosis variable would then be coded as not-depressed = - 1 and depressed = 1.
Note. When N is very small (e.g., below 20) and any expected cell frequencies are less than 5, a more accurate way to obtain r is through a several step process based on Fisher's Exact Test. Specifically, we a) compute Fisher's Exact Test, b) obtain the one-tailed p value associated with this test, c) convert the p value to a t value, and then d) compute r from that t and the corresponding degrees of freedom ( df = N – 2). These steps are explained and illustrated below.
a) Fisher's Exact Test is produced by the chi-square facility in SPSS or SAS (which can also be used to compute phi directly). Using the initial data set described above (i.e., with the raw data for our two variables, "dichot_a" and "dichot_b") we run the following syntax commands.
In SPSS , the syntax is as follows. CROSSTABS /TABLES= dichot_a BY dichot_b
/FORMAT= AVALUE TABLES /STATISTIC= CHISQ PHI CORR /CELLS= COUNT.
In SAS , the syntax would be as follows. PROC FREQ; TABLES dichot_a*dichot_b / CHISQ; RUN;
b) To obtain the one-tailed p from the Fisher's Exact Test output, it may be necessary to change the default output settings in SPSS or SAS so that p is reported to at least 2 non-zero digits. In other words, r cannot be obtained if the output reports " p < .000". Instead, we must obtain the actual p value to at least two meaningful digits (e.g., p = .000054 or p = .000000013).
c) Once the correct p value is obtained, we convert this value to t. (Note that in this instance, it is more accurate to convert p to t than it is to convert p to Z , the standard normal deviate.) To do so in SPSS or SAS, we use the inverse distribution function for t. Several steps are required. First in a new dataset create a variable labeled "p_by_FET" (to indicate it is for the p obtained from Fisher's Exact Test) and a second variable labeled "df" (to indicate degrees of freedom). Next, enter the observed p value from the Fisher's Exact Test in the p_by_FET column and enter N - 2 in the df column. Finally, run the following syntax.
In SPSS : COMPUTE t_by_p = IDF.T((1 - p_by_fet),df). EXECUTE.
In SAS : DATA newdata; SET olddata; t_by_p = TINV((1 - p_by_fet,df); RUN;
d) Next, we compute r from the t value just obtained. For this step, we use the same data set and just run one additional command. The command takes the t value from Step "c" and transforms it to r. The following syntax commands are used.
In SPSS: COMPUTE r_by_t = sqrt((t_by_p2) / ((t_by_p2) + df)). EXECUTE.
In SAS: DATA newdata; SET olddata; r_by_t = sqrt((t_by_p2) / ((t_by_p2) + df)); RUN;
PROC CORR; VAR cntrst_v dimen_v; RUN;
Note. The sign of this correlation should be checked to ensure that the direction of the contrast weights is consistent with the direction of the dimensional variable. In the example given above, higher scores on resilience and better outcome (i.e., higher scores on the outcome contrast weights) would lead to a positive correlation, which in turn would indicate the findings were consistent with the initial hypothesis. Had the dimensional variable been a measure of ego deficiency, for example, the contrast weights should have been assigned in the opposite direction, with higher values indicating worse outcome.
It also should be noted that each group that forms the multi-category variable does not need to be assigned a unique contrast weight. For instance, in the running example, if the researcher's original hypothesis was that psychological resilience would only predict who improved in treatment but not who stayed the same or got worse, then the contrast weights would have been different. Under this hypothesis, the correct contrast weights would be worse outcome = - 1, unchanged outcome = - 1, and improved outcome = 2. Thus, when creating the recoded "cntrst_v" variable, the subjects who improved would be assigned a score of 2 and all the other subjects would be assigned a score of - 1. Note that in each instance the contrast weights sum to
Finally, note that when three or more categories are employed for one of our variables several Pearson's r s can be computed, including reffect size , rcontrast , rBESD , and ralerting (see Rosenthal, Rosnow, & Rubin, 2000). The r computed in this section, and throughout this document, is r effect size , a conservative^ r^ that does not partial out noncontrast between group variation. The coefficient r effect size is defined as the correlation between each individual's membership in a group or class on the multi-category variable and his or her score on the dimensional variable.
I-A-4: Calculating Pearson's r From a Design With One Variable That Has 2 or More Ordered Categories and One Variable That Has 3 or More Ordered Categories (i.e., an Omnibus Chi-Square Design with df > 1).
An omnibus chi-square design is one in which the chi-square table is larger than 2 x 2 (i.e., it has more than four cells). For instance, one variable could be dichotomous while the other variable has three categories (i.e., a 2 x 3 design with 6 cells), or one variable may have four categories and the other variable has three categories (i.e., a 4 x 3 design with 12 cells).
To compute a meaningful effect size from this kind of design, it must be possible to place the grouping categories for BOTH variables on a hypothesized continuum. If this can be done, then the process of computing r is straightforward. Specifically, the procedures described in the previous section (i.e., I-A-3) for a single multi-category variable are generalized and applied to both of the multi-category variables. Thus, all subjects would have to be assigned appropriate contrast weights for the first variable (e.g., labeled "cntrst_a") and also for the second variable (e.g., labeled "cntrst_b"). If the variables were not initially scored this way, they would have to be recoded so that each category was assigned its proper contrast weight. Once this has been done, the two variables are correlated.
In SPSS syntax : CORRELATIONS /VARIABLES= cntrst_a cntrst_b /PRINT=TWOTAIL SIG /MISSING=PAIRWISE.
In SAS syntax : PROC CORR; VAR cntrst_a cntrst_b; RUN;
Note. The sign of this correlation should be checked to ensure that the correlation is positive when the direction of the result is consistent with the a priori hypothesis. This is easily accomplished by assigning the contrast weights for both variables in such a way that the hypothesized association leads to a positive correlation.
I-B. Cohen's d as the Effect Size
I-B-1: Calculating Cohen's d From a Design With a Dimensional Variable and a Dichotomous Variable (i.e., a t- Test Design).
To compute Cohen's d from a t- test design, in which there is one dichotomous variable and one dimensional variable, the first step is to compute descriptive statistics for the dimensional variable (dimen_v) within each category of the dichotomous variable (dichot_v).
This can be accomplished in a variety of ways. In SPSS one way is to use the "Split File" function to identify the two groups formed by dichot_v and then request means and standard deviations (or variances) for each group through the "Frequencies" command. SAS achieves the same purpose using the “By” command within “Proc Means.”
The SPSS syntax is as follows. SORT CASES BY dichot_v. SPLIT FILE SEPARATE BY dichot_v. FREQUENCIES VARIABLES= dimen_v /FORMAT=NOTABLE /STATISTICS=MEAN STDDEV VARIANCE /ORDER= VARIABLE.
The SAS syntax is as follows. PROC MEANS MEAN STD CSS; BY dichot_v; VAR dimen_v; RUN;
For the second step, the descriptive statistics given in the computer output would be placed into a new data file. The two categories for the dichotomous variable should be given meaningful
the final pooled ^2 is obtained by dividing the sum of the weighted variances by the sum of the weights.
Note. The sign of d is arbitrary. When the direction of the result is consistent with the a priori hypothesis, it should be reported with a positive sign. When a result is in the direction opposite of that specified by the hypothesis, it should be given a negative sign. This is easily accomplished by designating the group that is expected to have the higher mean as group 1.
The steps listed above are somewhat complicated. Thus, until SPSS and SAS introduce a command for generating Cohen's d , it is actually easier to compute Cohen's d after running a t- test to compare mean differences across groups. The computations using this method are described below in Section II-B-1.
SECTION II: COMPUTING EFFECT SIZES FROM THE OUTPUT OF STATISTICAL
TESTS AND TRANSLATING ONE EFFECT SIZE TO ANOTHER.
[ Note. Many of the following computations can be done using a hand calculator and are even easier to accomplish that way. However, syntax commands are still used to illustrate the computations.]
II-A. The Pearson Correlation as the Effect Size
II-A-1: Pearson's r From t- Test Output Comparing Means Across Two Groups.
To compute r from the output of a t- test, first create a data file that contains two variables, one labeled "t_value" and the other labeled "df". Next, enter the observed value for the t statistic in the "t_value" column and enter the degrees of freedom for the t- test in the "df" column. Finally, run the following formula (where sqrt = square root, and **2 = squared).
SPSS : COMPUTE r_by_t = sqrt((t_value2) / ((t_value2) + df)). EXECUTE.
SAS : DATA newdata; SET olddata; r_by_t = sqrt((t_value2) / ((t_value2) + df)); RUN;
Note. When converting t to r , it is important to use the standard pooled variances t - test results, not the results that correct for unequal variances. If an original research article only reported the t - value and df from the unequal variances formula, a reasonable estimate of r can be obtained by using the reported t - value in conjunction with the correct df , namely df = N - 2. If we instead used the adjusted df from the unequal variances formula it would produce an artificially inflated estimate of r.
Note. The r produced by this formula is reffect size when the t - test is derived from the traditional 2- group independent samples t - test or paired samples t - test. However, as discussed in Section II-A- 4, a t - test can also be used to examine the statistical significance of a focused contrast computed across three or more groups. In the latter instance, the r obtained by this formula is called rcontrast and it must be transformed further to produce reffect size. See Section II-A-4 for a more complete explanation.
Note. The sign of this correlation is arbitrary. When the direction of the result is consistent with the a priori hypothesis, it should be reported with a positive sign. When a result is in the direction opposite of that specified by the hypothesis, it should be given a negative sign.
Note. The sign of this correlation is arbitrary. When the direction of the result is consistent with the a priori hypothesis, it should be reported with a positive sign. When a result is in the direction opposite of that specified by the hypothesis, it should be given a negative sign.
II-A-4: Pearson's r From F Test Output When More Than Two Groups Have Been Compared.
It is more complicated to compute r from the output of an F test in a traditional ANOVA design, in which a dimensional variable has been compared across more than two groups (and it is not possible to compute Cohen's d from data of this type). Because of the increased complexity, example syntax for SPSS and SAS is not provided below. However, the procedures are described in the following sources and they are summarized and illustrated here. The material that follows presumes that readers are already familiar with previous sections in this document, particularly the discussion of contrast weights in Sections I-A-3 and I-A-4.
Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (2000). Contrasts and effect sizes in behavioral research: A correlational approach. New York: Cambridge University Press. Rosnow, R. L., & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counternulls on other people's published data: General procedures for research consumers. Psychological Methods, 1 , 331-340. Rosnow, R. L., Rosenthal, R., & Rubin, D. B. (2000). Contrasts and correlations in effect-size estimation. Psychological Science, 11 , 446-453.
The complexity of computing reffect size from the output of a traditional ANOVA arises from two main factors. First, obtaining the final coefficient requires multiple computational steps. Second, complexity arises from the fact that r can be computed from different kinds of incompletely reported initial information. Because ANOVA results in the published literature may report different types of incomplete information, the latter provides secondary researchers with some flexibility when they wish to compute reffect size from studies conducted by other people.
Several approaches to computing r will be explained and illustrated here. Each scenario makes uses of slightly different pieces of initial information. However, for simplicity, the same example data set is used throughout. The data set is described in Rosnow and Rosenthal (1996).
II-A- 4 - a: Scenario 1 - When given group M s and F or t from a focused contrast
Scenario 1 is applicable when the original output provides a) the mean for the dimensional variable in each of the multi-category groups and b) the t - value or F - value that comes from testing a focused contrast across the multi- category variable.
This is a basic scenario, although it is not likely to be encountered often because authors typically have computed omnibus F tests rather than focused contrasts. An omnibus F test determines whether there is any pattern in the means that is unlikely to be due to chance, while a focused contrast determines whether there is a specified and expected pattern in the means that is
unlikely to be due to chance. A meaningful effect size can be obtained from the latter but not the former.
Note. Either a t - test or an F test can be used to evaluate a multi-group focused contrast. These statistics are psychometrically equivalent because F = t^2. For instance, a researcher wishing to produce a focused contrast in SPSS would use the One-way ANOVA procedure and would specify contrast weight coefficients on the "/CONTRAST" subcommand. The statistical output from this set of commands would be in the form of a t - test rather than an F test.
Assuming an existing study provides mean scores across all groups and t or F from a focused contrast we can compute reffect size in three steps. First, the t or F is converted to rcontrast. Second, the coefficient called ralerting is computed. Third, rcontrast and ralerting are used to produce reffect size.
For the first step, use the formulas from Sections II-A-1 (for t - test output) or II-A-3 (for F test output) to translate the results of the statistical test into an initial r. Previously, the formulas in Sections II-A-1 and II-A-3 were used to compute reffect size directly from the t or F output. This was possible because the statistical results had been obtained by comparing means across just two groups. However, when t or F are obtained from a focused contrast that compares means across three or more groups, the r produced by these formulas is no longer equivalent to reffect size. Instead, the r is known as rcontrast to indicate that it is derived from a multi-group contrast.
The definition of rcontrast is somewhat difficult to grasp. Fortunately, a complete understanding is not essential to follow the procedures outlined here. Nonetheless, rcontrast is defined as the correlation between each individual's classification on the multi-category variable and his or her score on the dimensional variable AFTER partialing out between group variability that is not due to the focused contrast.
For the second step in our efforts to produce reffect size we need to compute ralerting , which is the correlation between each group's mean score on the dimensional variable and the contrast weight assigned to that group. Thus, if there are three groups with means of 2.0, 3.5, and 5.0 and the groups have been assigned contrast weights of - 1, 0, and 1, then these three pairs of values (i.e., 2.0 and - 1; 3.5 and 0; and 5.0 and 1) are correlated to produce ralerting = 1.0.
For the third and final step, the results from the first two steps are entered into the following formula to compute reffect size. (Note that both rcontrast and ralerting are squared in the denominator of the equation.)
reffect size = rcontrast / square root[(1 - r^2 contrast ) + ( r^2 contrast / r^2 alerting )]
To illustrate these procedures, suppose a researcher hypothesizes higher doses of anti-anxiety medication will lead to increased functioning in patients with anxiety disorders. The researcher randomly assigns 20 patients to four groups that differ in medication dose, including no medication, 100 milligrams, 200 mg, and 300 mg. Given these four conditions and the stated hypothesis that mean scores of functioning should increase in tandem with increasing medication dosage, appropriate contrast weights are assigned to the four groups to test a focused hypothesis. In this instance, appropriate contrast weights would be - 3, - 1, +1, and +3, respectively. These weights sum to zero and they indicate that the group means should be ordered as follows: no
Most often, findings in the published literature do not report F - tests derived from a focused contrast. A more common scenario occurs when the original report provides the mean, the SD, and the n for the dimensional variable in each of the multi-category groups. Using this information, we can compute reffect size in a 4 - step process. First, we compute a t test that evaluates the relevant focused contrast, known as tcontrast. This is the hard part. With this value in hand, we simply follow the remaining steps that had been outlined under Scenario 1. Thus, step two converts tcontrast to rcontrast , step three computes ralerting , and step four uses rcontrast and ralerting to obtain reffect size.
As mentioned, the hard part is computing a t - value that corresponds to the focused contrast. This is accomplished through a formula provided in Rosnow and Rosenthal (1996; Equation 1) or Rosenthal et al. (2000; Equation 3.10). This formula is somewhat difficult to reproduce in a text file that will be converted by various word processing programs. However, it is provided below in a somewhat longhand form.
tcontrast = Summation( Mi * CWi ) / square root[ MS within * (Summation( CW^2 i / ni ))]
where Mi is the mean for each group, CWi is the contrast weight assigned to each group, ni is the number of subjects in each group, and MS within is the error variance from a between subjects ANOVA. The MS within can also be estimated by computing the pooled variance across groups, with each variance weighted by sample size; i.e., [Summation(SD^2 i * ni ) / N ].
To illustrate this formula, we again use the example begun in Scenario 1. As before, the hypothesis is that functioning should increase as medication dosage increases. Although the original research report may not have specified appropriate contrast weights, the hypothesis dictates the appropriate contrast weights that we should assign to the four medication groups. Let's assume that this time the research report provided group M s, SDs and n s, but did not provide the results from an F test examining a focused hypothesis. Consequently, the available information consists of group data and a logical hypothesis that we use to produce contrast weights. The information is as follows:
Condition Post Treatment Contrast Mean SD n Weight No medication 3.0 1.0 5 - 3 100 mg 1.0 1.0 5 - 1 200 mg 9.0 1.0 5 + 300 mg 7.0 1.0 5 +
With these pieces of data, the first step is to compute tcontrast. However, to do so we also need the MS within. Because the research report did not provide this information, we will estimate it using the pooled variance across groups. Because each group has SD = 1 and n = 5, some readers may realize immediately that the pooled variance will be 1.0. However, for the sake of clarity, the computations will be illustrated across the four groups using the equation Summation(SD^2 i * ni ) / N , where i = groups 1 to 4.
MS within = Summation(SD^2 i * ni ) / N = [(1^2 * 5) + (1^2 * 5) + (1^2 * 5) + (1^2 * 5)] / 20 = (5 + 5 + 5 + 5) / 20 = 20 / 20 = 1.
Given that we now know MSwithin , the tcontrast computations can continue:
tcontrast = Summation( Mi * CWi ) / square root[ MS within * (Summation( CW^2 i / ni ))] = [(3-3)+(1-1)+(91)+(73)] / square root[1.0*((- 32 /5)+(- 12 /5)+(1^2 /5)+(3^2 /5))] = [-9 + - 1 + 9 + 21] / square root[1.0 * (1.8 + 0.2 + 0.2 + 1.8)] = 20 / square root(4) = 20 / 2 = 10.
(This t result is the same as that obtained in Scenario 1 when we computed t by taking the square root of the F value that the researcher had reported for testing the focused contrast.)
For step two, we use the formula from Section II-A-1 to translate this tcontrast into rcontrast. Because the tcontrast in this instance is computed from a study with 20 subjects in a 4-group design, dfwithin = 20 - 4 = 16.
rcontrast = square root( t^2 / ( t^2 + dfwithin )) = square root(10.0^2 / (10.0^2 + 16)) = square root(100 / 116) = square root(.862) =.
Thus, as in Scenario 1, the result is rcontrast = .928.
For step three, we compute ralerting by correlating the four group means (3.0, 1.0, 9.0, and 7.0) with their assigned contrast weights (-3, - 1, 1, and 3, respectively). Doing so we obtain ralerting = .707.
For the fourth and final step, we plug the values for rcontrast and ralerting into the reffect size formula.
reffect size = rcontrast / square root[(1 - r^2 contrast ) + ( r^2 contrast / r^2 alerting )] = .928 / square root[(1 - .861) + (.861 / .500)] =.
Thus, using slightly different initial information, we once again see that the researcher's focused hypothesis was supported and increasing dosages of anti-anxiety medication are associated with improved functioning at a magnitude of reffect size = .68.
F MPC = FOmnibus * dfbetween = 66.667 * 3 = 200.
For the second step, we compute the alerting correlation, ralerting , which is the correlation between the four group means (3, 1, 9, and 7) and their contrast weights (-3, - 1, 1, and 3, respectively). As before, we obtain ralerting = .707.
Third, we compute the F - value that corresponds to our hypothesized contrast, which is designated Fcontrast. This is computed according to the following formula, which is then illustrated using the example data.
Fcontrast = F MPC * r^2 alerting = 200.00 * (.707^2 ) = 200.00 *. = 100.
Fourth, we translate the Fcontrast into rcontrast by using the formula in Section II-A-3 of this document. Doing so for the example data produces rcontrast = .928, as illustrated below.
rcontrast = square root( F / ( F + dfwithin )) = square root(100 / (100 + 16)) = square root(.86) =.
Fifth, and finally, we insert the rcontrast and ralerting values into the formula for reffect size.
reffect size = rcontrast / square root[(1 - r^2 contrast ) + ( r^2 contrast / r^2 alerting )] = .928 / square root[(1 - .861) + (.861 / .500)] =.
Thus, once again, when using slightly different initial information, we see that increasing dosages of anti-anxiety medication are associated with improved functioning because reffect size = .68.
II-A-5: Pearson's r From the Output of an Omnibus Chi-Square Test (i.e., df > 1).
It is also more complicated to compute r from the output of an omnibus chi-square analysis, in which one variable has 2 or more categories and the other variable has 3 or more categories. Because of the increased complexity, example SPSS and SAS syntax is not provided below. However, basic procedures are described most explicitly in Rosnow and Rosenthal (1996) and they are summarized here. After summarizing Rosnow and Rosenthal's procedures, a computationally simple alternative procedure is also described.
Rosnow, R. L., & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counternulls on other people's published data: General procedures for research consumers. Psychological Methods, 1 , 331-340.
II-A- 5 - a: The Rosnow and Rosenthal (1996) Approach
The procedures outlined in Rosnow and Rosenthal (1996) apply to a chi-square in which one variable has just two categories and the other has three or more categories (i.e., a 2 x k table, where k = columns > 2). In what follows, the data will be organized so the two-category variable constitutes the rows and the variable with three or more categories constitutes the columns. The procedures require that the existing output contain frequencies for each cell in the chi-square table or proportions that can be converted to frequencies. Based on this information, to compute reffect size we follow a seven-step process.
- Using theory or prior research, we determine appropriate contrast weights ( CWi) for each column (i.e., for each category in the variable that has 3 or more categories).
- We then select one of the two rows to be a reference row and compute pi , which is the proportion of observations that fall in the cells formed by the intersection of that row and each column. Specifically, pi is computed as the cell frequency ( fi ) divided by its column total ( ni ). pi = fi / ni
- Next, we compute the variance for each column proportion, pi , using the following formula. S^2 i = pi * (1 - pi ) / ni
- We then compute Zcontrast according to the following formula, with notation as defined in the previous steps. Zcontrast = Summation( pi * CWi ) / square root[(Summation( S^2 i * CWi ))]
- Next, we translate the Zcontrast to rcontrast using the following formula, where N = total sample size. rcontrast = Zcontrast / square root( N )
- Next, we compute ralerting , which is now the correlation between each column proportion ( pi ) and its corresponding contrast weight ( CWi ).
- Finally, we use ralerting and rcontrast to compute reffect size via the formula provided earlier. reffect size = rcontrast / square root[(1 - r^2 contrast ) + ( r^2 contrast / r^2 alerting )]
Because Rosnow and Rosenthal's (1996) emphasis was on explaining how to compute focused contrasts, rather than effect sizes, steps 6 and 7 are not detailed in their article. They are also not described in the other sources we have been referencing (i.e., Rosenthal et al., 2000; Rosnow et al., 2000). Nonetheless, to compute ralerting from a chi-square table having 2 rows and k columns, one selects the cell proportions in one of the rows to serve as reference values, which makes them analogous to the condition means that we have used previously when computing ralerting. (In fact, if entries in the first row of the table are coded as 0 and entries in the second row are coded as 1, the proportions for each cell in the second row are the means for each column.)
As can be seen, there are many steps and formulas that must be used to compute reffect size from a 2 x k table according to these procedures. In addition, the procedures are limited to instances when the original data relied on one variable that was dichotomous.
