Download Research Methodologies and Statistical Concepts and more Exams Career Counseling in PDF only on Docsity!
CPCE: Research and Program Evaluation Exam 77 Questions
with Answer
Research vs. Evidence Based Inquiry - CORRECT ANSWER Research: the systematic process of collecting and analyzing data for some purpose such as investigating a problem or answering a question. Evidence-based inquiry: the search for knowledge using empirical data which has been gathered systematically. Quantitative Research - CORRECT ANSWER Assumes social facts have a single objective reality. Tends to study samples or populations. Researchers try not to influence collection of data (instruments). Statistical methods comparing and contrasting groups occurs. Researchers examine for causes and relationships. Qualitative Research - CORRECT ANSWER Assumes multiple realities socially constructed by individuals and groups. Tends to study individual units - person, family, community — in naturalistic setting. Researchers may be primary instrument for collecting data (through observation). Researchers' impressions, judgments and feelings may be used. Goal is to describe the nature of things.
Inductive vs. Deductive Research - CORRECT ANSWER Inductive: this research begins at the real world, practical level. It tends to be descriptive, correlational, or historical and leads to the building of theory. Deductive: this research springs from theory which is already established. This research tries to determine what the relationships are between elements of the theory and may be experimental in nature. Quantitative (Non-Experimental Designs): Survey - CORRECT ANSWER This may occur through questionnaires, interviews, etc. and is used to measure attitudes, perceptions, etc. (For example: Public Opinion Poll). Often the response rate of survey research is low, below 50 percent. Unless you know that the characteristics of the non- respondents are similar to the characteristics of the respondents, you must be cautious in generalizing. Quantitative (Non-Experimental Designs): Descriptive - CORRECT ANSWER Descriptive: this research simply describes an existing state of events. Numbers may be used to characterize groups or individuals. Quantitative (Non-Experimental Designs): Comparative - CORRECT ANSWER This research method investigates whether there are differences between two or more groups. There is no manipulation of conditions experienced by each group. Quantitative (Non-Experimental Designs): Correlational - CORRECT ANSWER This research method uses the correlation coefficient to determine the degree of relationship between two or more variables or phenomena. Quantitative (Non-Experimental Designs): Ex Post Facto - CORRECT ANSWER Also called causal-comparative, this research design studies possible causal relationships among variables ex post facto (after the fact). You do not manipulate any variables; the focus is on what has already happened (after the fact). You may generate 'several' reasons (causes) for the relationships you discover.
Qualitative Research: Interactive - CORRECT ANSWER i. Case study in which the case may be a program, activity, or a set of individuals who are bounded in time and place. ii. Ethnography which is a description and interpretation of a cultural or social group or system. Data is typically collected through observation and interviewing. The issue of observer bias is important. Qualitative Research: Noninteractive - CORRECT ANSWER This is analytical research conducted primarily through document analysis. Examples might be historical analysis (collecting and analyzing documents describing former events), biographical analysis (written or oral) and legal analysis which focuses on law and court decisions. Mixed-Methods Research Design - CORRECT ANSWER These designs combine quantitative and qualitative methods in the same research effort. The researcher retains the flexibility to use both types of designs. Typically, the designs are used sequentially. For example, quantitative may be gathered first and then qualitative methods are used to further explain or elaborate on the findings, using perhaps, surveys, interviews, or focus groups. Single-Subject Design - CORRECT ANSWER Studies the effects of a program or treatment on an individual or group treated as an individual, usually after a baseline has been established. Action Research - CORRECT ANSWER Conducted in an attempt to improve services or a program. This research may often be viewed as having an evaluative function. Pilot Study - CORRECT ANSWER A small-scale research effort often used to determine the feasibility of a large scale effort with emphasis on refining procedures and instrumentation.
Longitudinal Research - CORRECT ANSWER Consists of collecting data from the same group of individuals over a period of time. This is also called a panel study. For example, this author studied the career development of school children beginning when they were second graders and re-interviewing them every two years until they were high school seniors. Cross-Sectional Research - CORRECT ANSWER Consists of collecting data from different groups at the same time and examining these differences. For example, one approach to studying career development is collecting data from each grade of students, for example from second to twelfth grade, at the same time. Within-Subjects vs. Between-Subjects - CORRECT ANSWER Within-subjects: examining what changes occur within the members of a group. Between subjects: examining what changes occur between two or more groups. Meta-Analysis - CORRECT ANSWER This is research comparing findings across studies, i.e., the results of many studies are examined simultaneously and one or more research questions answered. For example, a number of studies have examined what works in various counseling situations. Examining a number of these studies simultaneously can help determine what counseling techniques work for what kind of clients, with what kinds of problems, under what conditions. Internal Validity - CORRECT ANSWER Experiments are internally valid to the extent that extraneous variables have been controlled. In other words, to the extent that the treatment variable is the only one producing the observed changes, the experiment is internally valid. Threats to Internal Validity: Selection of Subjects - CORRECT ANSWER Differences in the results between two groups may not be due to the treatment variable experienced by one group because the composition of the two groups are different to begin with (probably not randomly selected).
There are several threats to the external validity of experiments and some of these are also threats to internal validity. Threats to External Validity: Selection of Subjects - CORRECT ANSWER If subjects are not randomly selected, the results may only apply to the subjects in the study. Furthermore, the results can only be generalized to people with similar characteristics. Can we generalize results of research with college students to other adults in general? Threats to External Validity: Ecological Validity - CORRECT ANSWER The research has ecological validity if the results can be generalized from one setting or circumstance to another. Sometimes the circumstances, conditions or physical surroundings of the research are so unique that the results cannot be generalized beyond that study. Threats to External Validity: Subject Reactions (Reactivity) - CORRECT ANSWER If any of the following are present, the results may not generalize beyond the study: i. Hawthorne effect -refers to the influence in performance which occurs when subjects receive attention or know they are participating in research. ii. Demand characteristics -- these are all the cues, information, knowledge, even rumors the subject has heard about the experiment which are likely to influence his or her performance. iii. Experimenter bias -- these are the changes in the subject's behavior brought about by the researcher's expectations, behaviors or attitudes. It may also be called the Rosenthal effect. He conducted research into this phenomenon and called it the Pygmalion effect which refers to the self- fulfilling expectation of doing well because it is expected. iv. Placebo - broadly, the placebo refers to any control treatment. The placebo should be identical to the experimental treatment except for the critical item being studied (new drug, parenting skills training, reaction to violence in films, etc). Even so, control subjects may be influenced by the placebo and react in unintended ways. Threats to External Validity: Novelty and Disruption Effect - CORRECT ANSWER The measured effect of the treatment on the subjects may be due to its novelty or
the disruption it causes. Being selected for a research project may be exciting and energizing; as it continues it may begin to disrupt routine and one's typical schedule. When novelty and disruption wear off or stabilize, there may be no long-term effects of the treatment. Levels of Measurement - CORRECT ANSWER a. Nominal: the numbers represent the variable's qualities or categories. With nominal data, you would ordinarily use a nonparametric statistic such as chi-square. b. Ordinal: the numbers represent differences in some magnitude of the variable. You can order the data, from top to bottom, from high to low. For example: Scores on a Psychology 101 final exam can be ranked from highest to lowest. c. Interval: the intervals between the numbers on a scale contain the same amount of the variable throughout the scale. These intervals provide a constant unit of measurement. For example: On a standardized test, the distance (interval) between 11 and 12 is the same as the distance between 24 and 25. Other examples are Fahrenheit or centigrade temperature scales. d. Ratio: the numbers are on a scale which has a true zero. In addition the numbers can be compared by ratios. For example: Someone who weighs 200 pounds is twice as heavy as someone who weighs 100 pounds. In counseling and psychology, we cannot say someone is twice as introverted as someone else. Random Sampling - CORRECT ANSWER All the individuals in the population have an equal and independent chance of being selected. Stratified Sampling - CORRECT ANSWER This refers to selecting in such a way that major subgroups in the population will be sampled. These subgroups may be based on ethnicity, gender, age, etc. Proportional Stratified Sampling - CORRECT ANSWER This refers to randomly selecting the same proportion of individuals for the sample as they represent proportionally in the major subgroups in the population. For example: If one-half of a population is Hispanic and one-half is white, you would randomly select your sample to be one-half Hispanic and one-half white.
normally distributed. You have para (two-sided) data that yields a bell-shaped curve. Also, you assume that the variance of the sample you are studying is homogeneous (similar) to the variance of the population from which your sample is drawn. The scores would give you a normal curve -- more or less. Examples of parametric statistics are: i-test and analysis of variance. Nonparametric: used when you cannot make any assumption about the shape of the curve or variance of the population scores (that is, they may not be normally distributed and variances may not be homogeneous). You want to make a statement about a school's academic performance but you only collected scores from the "gifted and talented" students. You will not have a normal distribution of academic scores for the school. Examples of nonparametric statistics are: Chi-square, Mann-Whitney U Test, and Wilcoxon Signed-Ranks Test, all to be reviewed later. Independent Variable - CORRECT ANSWER This is the variable you manipulate or vary to see what change occurs in the dependent variable. Sometimes you group or categorize the independent variable. For example, you categorize a group of individuals by gender, male and female. Or, you categorize high school students as freshmen, sophomores, juniors, and seniors. The independent variable precedes or is antecedent to the dependent variable. The independent variable is sometimes called: stimulus variable, predictor variable or experimental variable. Dependent Variable - CORRECT ANSWER This is the variable you are measuring or trying to change. The value of this variable depends upon the value of the independent variable you selected. The dependent variable is sometimes called: response variable, outcome variable or criterion variable. Research Questions and Hypotheses - CORRECT ANSWER Some investigators ask research questions to be answered. Some investigators formulate hypotheses (often in the null form) to test. The null hypothesis states there is no difference between the variables or groups measured. Ex: There are no significant differences in final academic grades (GPA) between boys and girls finishing the tenth grade.
One alternative hypothesis (directional) states that one group's score will be significantly different from another group's score (one-tailed test). Ex: The GPA of girls will be higher than the GPA of boys finishing the tenth grade. Another alternative hypothesis (nondirectional) states that there will be differences between the groups but which group has higher or lower scores is not indicated (two-tailed test). Ex: The GPA of girls and boys finishing the tenth grade will be different. Significance Level - CORRECT ANSWER There is always a possibility that in regard to your research results and the null hypothesis you posed, you will make a mistake in rejecting the null hypothesis when you should not, or in accepting the null hypothesis when you should reject it. The significance level you select will determine your likelihood of making either mistake. The significance level may be set by you before data analysis at .05, .01 or some other level such as .001. The level selected is your willingness to make an error, that is, rejecting the null hypothesis, when in fact there is not a significant difference between the groups. (Conventional significance levels used in research are .05, .01. and .001.) At .05, you are willing to accept the possibility of rejecting the null hypothesis in error five times out of one hundred times (if you performed the experiment 100 times). Type I (Alpha) Error - CORRECT ANSWER Type I or alpha error are the names we use in referring to the rejection of the null hypothesis (which states that there is no difference) when it is correct. In short, you make an error. Remember: at a significance level of .05, you will reject the null hypothesis five times out of 100 when in fact there is not a real difference. You can change the probability of Type I error by changing your significance level. Type II (Beta) Error - CORRECT ANSWER Type II error refers to the failure to reject the null hypothesis when there is, in fact, a difference. As the significance level goes down (e.g., .05 to .01), Type I error decreases but Type II error increases, i.e., the failure to reject the null hypothesis when you should. As one type of error goes up, the other goes down and vice versa.
more variables (factors) differ significantly from each other and whether the factors interact significantly with each other. In our example in Item 18, we are comparing self-esteem scores of clients who have been counseled by female and male counselors. Let's assume we want to compare self-esteem scores with a second counselor variable as well, kind of training. Our two independent variables then are (I) gender (two kinds -female and male) and (2) kind of training. Let's stay with the same three kinds of training we used in Item 22. The one dependent variable is the measure of self-esteem. This IS a 2 X 3 ANOVA design. By completing this calculation, one could deter- mine if there are any significant differences in client self-esteem based on therapist gender, or kind of therapist training, or whether there is a significant interaction between gender and training. This is called a factorial analysis of variance because the effects of two or more factors (independent variables) are being measured. ANOVA: Multivariate (MANOVA) - CORRECT ANSWER When your analysis involves more than one dependent variable, you cannot use Factorial ANOVA. The appropriate statistic is multivariate analysis of variance (MANOVA). If you were measuring something else about the clients in addition to self-esteem, that would be more than one dependent variable. In our continuing example (See Item 23 - ANOVA), we had two independent variables - gender and kind of training. Our single dependent variable was a measure of self-esteem. Let's assume that we also want to measure the locus of control (internal vs. external orientation) of clients when they conclude therapy. We have each client complete a locus of control scale in addition to the self- esteem inventory. Whenever you have two or more dependent variables, you cannot use ANOVA. You must use multivariate analysis of variance (MANOVA). Analysis of Covariance (ANCOVA) - CORRECT ANSWER Analysis of covariance is used similarly to analysis of variance except that the influence of one or more independent variables on the dependent variable is controlled. This means that initial group differences are adjusted statistically on one or more variables that are related to the dependent variable.
For example: In our continuing example (Item 18 and onward), we are looking at the impact of therapists' gender on clients' self-esteem. Let's assume that we do not have random assignment of clients to therapists. In fact, let's assume those most female clients are assigned to female counselors and most male clients are seen by male counselors. On the pretest of self-esteem, we find that women clients score an average of five points higher than male clients. In ANCOVA, we make use of such information which is obviously related to our dependent measure, self- esteem. If we were measuring the impact of therapist gender on clients' self- esteem, women therapists would have started out five points higher on client self-esteem. So we statistically adjust client self-esteem scores (for example, add points to the males pretest scores and subtract points from the female pretest scores) before making the comparison. In this case, the covariate is the systematic difference in self-esteem scores on the pretest (women clients score an average of five points higher than men clients). Consequently, we conduct an analysis of covariance - adjust the scores. Then we do the analysis of variance. Post Hoc or Multiple Comparisons Test - CORRECT ANSWER If your analysis of variance yields a significant F value, you still will not know which particular pair of mean scores is significantly different from each other. You must apply one of the post hoc (after the fact) tests to determine whether different particular group means or combinations of group means are significantly different. The most commonly used post hoc or multiple comparison tests are: Scheffe's (most conservative) Tukey's HSD (Honestly Significant Difference) Newman-Keuls Duncan's new multiple range test Your computer data analysis program will conduct these automatically. Nonparametric Tests - CORRECT ANSWER When you cannot assume that your distribution of scores is normally distributed (resembles a normal curve) or that the variance of your sample is similar to the variance of the population (homogeneity) you must use nonparametric statistics.
Multiple Regression - CORRECT ANSWER This is the use of the correlation coefficient to determine the strength of the relationship of predictor (independent) variables on a criterion (dependent) variable. Multiple regression adds together the predictive power of several independent variables (predictors). For example: Predictor variables such as high school GPA, class rank, and ACT scores may be used to predict the criterion (outcome) variable, which could be end-of-college freshman year GPA. Scatterplot - CORRECT ANSWER This is a graphic representation of the relationship between two variables for a group of individuals. Each 1 on the graph is an individual who receives a score on the X test (e.g., anxiety) and a score on the Y test (e.g., hostility). The following statement might represent the data; As individuals get more anxious about the counselor exam, their hostility increases (positive relationship). Factor Analysis - CORRECT ANSWER This is a statistical method using the correlation coefficient to determine whether a set of variables can be reduced to a smaller number of factors. For example: A factor analysis of all the items on a long inventory with 15 scales may uncover only four or five factors independent of each other underlying the scales. Thus many of the constructs of the 15 scales overlap with each other. Likert Scale - CORRECT ANSWER This is a widely used technique for measuring attitudes or opinions. It allows for several response choices, e.g., a five (or seven or nine) point scale. Biserial Correlation - CORRECT ANSWER An appropriate correlation coefficient to use when one variable yields continuous data and the other yields data that is dichotomous. Degrees of Freedom - CORRECT ANSWER The number of observations that are free to vary.
Double-Blind Technique - CORRECT ANSWER This occurs when neither the researcher nor the subject knows who is getting the active substance or the placebo. Halo Effect - CORRECT ANSWER This is the tendency for the observer (researcher or data collector) to form an early impression of the person being observed and then letting this impression influence observations or ratings of that individual. Halo effect may be positive or negative. Heteroscedasticity - CORRECT ANSWER One end of a distribution of scores has more variability than the other end resulting in a fan-like appearance. Homoscedasticity - CORRECT ANSWER There is an equal distribution of scores throughout the range of scores, i.e., around a line of best fit. Inter-Rater Reliability - CORRECT ANSWER In qualitative research, the reliability calculated by correlating the responses of several raters. Observer Bias - CORRECT ANSWER This is the tendency of researchers to see, hear and remember what they want to. Placebo - CORRECT ANSWER Originally meant an inert drug (typically, a sugar pill); now it also means a control treatment that gives subjects the same amount and kind of attention as experimental group subjects get (this reduces Hawthorne and Rosenthal effects). Rank-Order Correlation (Spearman rho) - CORRECT ANSWER Used when the values of the variables are reported in rank form rather than continuous. Counseling Program Evaluation - CORRECT ANSWER The emphasis on accountability in the human services field accelerated in the 1970s and continues today. The push for accountability comes primarily from funders including Health Maintenance Organizations, insurance companies, government funding sources at all levels, and others. This accountability in human services reflects what the business world adopted many years ago as the 'bottom line.' There is an acute need to demonstrate the efficacy of counseling in general and the effectiveness of specific theories, techniques, and approaches in particular.
Writing Research - CORRECT ANSWER Most counseling and psychological journals require that the writing style presented by the Publication Manual ofthe American Psychological Association be used. Sexist language is to be avoided. Never submit a professional research manuscript to more than one journal for publication consideration at a time.
