Download Understanding Research Validity: External, Internal, Construct, and Conclusion and more Quizzes Communication in PDF only on Docsity!
Types of validity in
research
External validity Construct validity Internal validity Conclusion validity TERM 2
External
validity
DEFINITION 2 Generalizability. Externalvalidity refers to the approximate truth of conclusions the involvegeneralizations. Put in more pedestrian terms, external validity is the degree to which the conclusions in your study would hold for other persons in other places and at other times. TERM 3
Construct
validity
DEFINITION 3 The degree to which inferences can legitimately be made from the operationalizations in your study to the theoretical constructs on which those operationalizations were based. Likeexternal validity, construct validity is related to generalizing. But, where external validity involves generalizing from your study context to other people, places or times, construct validity involves generalizing from your program or measures to theconceptof your program or measures. You might think of construct validity as a "labeling" issue. TERM 4
Internal
validity
DEFINITION 4 The approximate truth about inferences regarding cause- effect or causal relationships. Thus, internal validity is only relevant in studies that try to establish a causal relationship. It's not relevant in most observational or descriptive studies, for instance. TERM 5
Conclusion
validity
DEFINITION 5 The degree to which conclusions we reach about relationships in our data are reasonable. Conclusion validity also pertains to causal relationships. How do we distinguish it frominternal validitywhich is also involved with causal relationships? Conclusion validity is only concerned with whether there is a relationship.
Theoretical
Concerned with developing, exploring or testing the theories or ideas that social researchers have about how the world operates TERM 7
Empirical
DEFINITION 7 Based on observations and measurements of reality -- on what we perceive of the world around us TERM 8
Nomothetic
DEFINITION 8 Laws or rules that pertain to the general case (nomos in Greek) and is contrasted with the term "idiographic" which refers to laws or rules that relate to individuals (idios means 'self' or 'characteristic of an individual ' in Greek). In any event, the point here is that most social research is concerned with the nomothetic -- the general case -- rather than the individual. TERM 9
Probabilistic
DEFINITION 9 Based on probabilities. The inferences that we make in social research have probabilities associated with them -- they are seldom meant to be considered covering laws that pertain to all cases. Part of the reason we have seen statistics become so dominant in social research is that it allows us to estimate probabilities for the situations we study. TERM 10
Causal
DEFINITION 10 Interested (at some point) in looking at cause-effect relationships.
Longitudinal study
One that takes place over time -- we have at least two (and often more) waves of measurement in a longitudinal design. TERM 17
Types of longitudinal designs
DEFINITION 17 Repeated measures Time series TERM 18
Repeated
measures
DEFINITION 18 Two or a few waves of measurement TERM 19
Time series
DEFINITION 19 Many waves of measurement over time TERM 20
Variable
DEFINITION 20 Any entity that can take on different values
Attribute
A specific value on a variable. For instance, the variablesexorgenderhas two attributes:maleandfemale. TERM 22
Exhaustive variable
DEFINITION 22 Include all possible answerable responses. For instance, if the variable is "religion" and the only options are "Protestant", "Jewish", and "Muslim", there are quite a few religions I can think of that haven't been included. The list does not exhaust all possibilities. TERM 23
Mutually exclusive variable
DEFINITION 23 No respondent should be able to have two attributes simultaneously. While this might seem obvious, it is often rather tricky in practice. For instance, you might be tempted to represent the variable "Employment Status" with the two attributes "employed" and "unemployed." TERM 24
Hypothesis
DEFINITION 24 A specific statement of prediction. It describes in concrete (rather than theoretical) terms what you expect will happen in your study. Not all studies have hypotheses. Sometimes a study is designed to be exploratory (seeinductive research). There is no formal hypothesis, and perhaps the purpose of the study is to explore some area more thoroughly in order to develop some specific hypothesis or prediction that can be tested in future research. A single study may have one or many hypotheses. TERM 25
Alternative hypothesis
DEFINITION 25 The hypothesis you support
Unit of analysis
The major entity that you are analyzing in your study. For instance, any of the following could be a unit of analysis in a study: individuals groups artifacts (books, photos, newspapers) geographical units (town, census tract, state) social interactions (dyadic relations, divorces, arrests) TERM 32
Hierarchical modeling
DEFINITION 32 Collecting data at one level (individual) and analyzing it at another (group) TERM 33
Two of the most important research fallacies
DEFINITION 33 Ecological fallacy Exception fallacy TERM 34
Ecological fallacy
DEFINITION 34 When you make conclusions about individuals based only on analyses of group data. For instance, assume that you measured the math scores of a particular classroom and found that they had the highest average score in the district. Later (probably at the mall) you run into one of the kids from that class and you think to yourself "she must be a math whiz." TERM 35
Exception fallacy
DEFINITION 35 When you reach a group conclusion on the basis of exceptional cases. This is the kind of fallacious reasoning that is at the core of a lot of sexism and racism. The stereotype is of the guy who sees a woman make a driving error and concludes that "women are terrible drivers."
Hourglass structure of research
Begin with broad questionsNarrow down, focus inOperationalizeObserveAnalyze dataReach conclusionsGeneralize back to questions TERM 37
Components of a causal study
DEFINITION 37 The Research Problem The Research Question The Program (Cause) The Units The Outcomes (Effect) The Design TERM 38
Deductive reasoning
DEFINITION 38 Wyysforks from the more general to the more specific. Sometimes this is informally called a "top-down" approach. We might begin with thinking up atheoryabout our topic of interest. We then narrow that down into more specifichypothesesthat we can test. We narrow down even further when we collectobservationsto address the hypotheses. This ultimately leads us to be able to test the hypotheses with specific data -- aconfirmation(or not) of our original theories. TERM 39
Inductive reasoning
DEFINITION 39 works the other way, moving from specific observations to broader generalizations and theories. Informally, we sometimes call this a "bottom up" approach. In inductive reasoning, we begin with specific observations and measures, begin to detect patterns and regularities, formulate some tentative hypotheses that we can explore, and finally end up developing some general conclusions or theories. TERM 40
Epistemology
DEFINITION 40 The philosophy of knowledge or of how we come to know.
Triangulation
Using multiple sources to help get a better idea of what's actually happening TERM 47
Incommensurability of perspectives
DEFINITION 47 The idea that we can never understand each other because we come from different experiences and cultures. TERM 48
Constructivism
DEFINITION 48 Belief that we each construct our view of the world based on our perceptions of it. Because perception and observation is fallible, our constructions must be imperfect. TERM 49
Objectivity
DEFINITION 49 Positivists believed that objectivity was a characteristic that resided in the individual scientist. Scientists are responsible for putting aside their biases and beliefs and seeing the world as it 'really' is. Post-positivists reject the idea that any individual can see the world perfectly as it really is. We are all biased and all of our observations are affected (theory-laden). Our best hope for achieving objectivity is to triangulate across multiple fallible perspectives! Thus, objectivity is not the characteristic of an individual, it is inherently a social phenomenon. TERM 50
Validity
DEFINITION 50 The best available approximation to the truth of a given proposition, inference or conclusion
The cause construct
A theory of what a cause might be TERM 52
The effect construct
DEFINITION 52 An idea of what we are ideally trying to affect and measure TERM 53
Voluntary participation
DEFINITION 53 Requires that people not be coerced into participating in research. This is especially relevant where researchers had previously relied on 'captive audiences' for their subjects -- prisons, universities, and places like that. TERM 54
Informed consent
DEFINITION 54 Prospective research participants must be fully informed about the procedures and risks involved in research and must give their consent to participate. TERM 55
Confidentiality
DEFINITION 55 Identifying information will not be made available to anyone who is not directly involved in the study.
3 major types of qualitative data
In-depth interviews Direct observation Written documents TERM 62
4 major qualitative approaches
DEFINITION 62 Ethnography Phenomenology Field research Grounded theory TERM 63
Ethnography
DEFINITION 63 The emphasis in ethnography is on studying an entire culture. Originally, the idea of a culture was tied to the notion of ethnicity and geographic location (e.g., the culture of the Trobriand Islands), but it has been broadened to include virtually any group or organization.However, the most common ethnographic approach isparticipant observationas a part of field research. The ethnographer becomes immersed in the culture as an active participant and records extensive field notes. TERM 64
Phenomenology
DEFINITION 64 A school of thought that emphasizes a focus on people's subjective experiences and interpretations of the world. That is, the phenomenologist wants to understand how the world appears to others. TERM 65
Field research
DEFINITION 65 Can also be considered either a broad approach to qualitative research or a method of gathering qualitative data. the essential idea is that the researcher goes "into the field" to observe the phenomenon in its natural state or in situ. As such, it is probably most related to the method ofparticipant observation. The field researcher typically takes extensive field notes which are subsequently coded and analyzed in a variety of ways.
Grounded theory
A complexiterativeprocess. The research begins with the raising ofgenerative questionswhich help to guide the research but are not intended to be either static or confining. As the researcher begins to gather data,core theoretical concept(s)are identified. Tentativelinkagesare developed between the theoretical core concepts and the data. This early phase of the research tends to be very open and can take months. Later on the researcher is more engaged in verification and summary. The effort tends to evolve toward onecore categorythat is central. TERM 67
Qualitative methods
DEFINITION 67 Participant observation Direct observation Unstructured interviewing Case studies TERM 68
Criteria for judging qualitative
validity
DEFINITION 68 Credibility Transferability Dependability Confirmability TERM 69
Research
DEFINITION 69 Thesystematic gathering ofinformation to answer aquestion or solve aproblem TERM 70
Types of research
DEFINITION 70 Formal / InformalAppliedTheoretical (basic)MethodologicalDescriptiveExploratoryExplanatory
Cluster (Area) random sampling
divide population into clusters (usually along geographic boundaries) randomly sample clusters measureallunits within sampled clusters TERM 77
Multi-stage sampling
DEFINITION 77 Combining sampling methods TERM 78
Statistic
DEFINITION 78 When we look across the responses that we get for our entire sample, we use a statistic. There are a wide variety of statistics we can use -- mean, median, mode, and so on. TERM 79
Parameter
DEFINITION 79 If you measure the entire population and calculate a value like a mean or average, we don't refer to this as a statistic TERM 80
Sampling distribution
DEFINITION 80 The distribution of an infinite number of samples of the same size as the sample in your study
Sampling error
We base our calculationon the standard deviation of our sample. The greater the sample standard deviation, the greater the standard error (and the sampling error). The standard error is also related to the sample size. The greater your sample size, thesmallerthe standard error. TERM 82
rule
DEFINITION 82 Start with the average -- the center of the distribution. If you go up and down (i.e., left and right)onestandard unit, you will include approximately 68% of the cases in the distribution (i.e., 68% of the area under the curve). If you go up and downtwostandard units, you will include approximately 95% of the cases. And if you go plus-and- minusthreestandard units, you will include about 99% of the cases. Notice that I didn't specify in the previous few sentences whether I was talking about standarddeviationunits or standarderrorunits. That's because the same rule holds for both types of distributions (i.e., the raw data and sampling distributions). TERM 83
Modal instance sampling
DEFINITION 83 Sampling the most frequent case, or the "typical" case. TERM 84
Expert sampling
DEFINITION 84 The assembling of a sample of persons with known or demonstrable experience and expertise in some area. TERM 85
Quota sampling
DEFINITION 85 There are two types of quota sampling:proportionalandnon proportional. In proportional quota sampling you want to represent the major characteristics of the population by sampling a proportional amount of each. Nonproportional quota sampling is a bit less restrictive. In this method, you specify the minimum number of sampled units you want in each category. here, you're not concerned with having numbers that match the proportions in the population.
Pct Across
Percent of that population that bought the product ie. % of people who graduated college bought snickers TERM 92
Pct Down
DEFINITION 92 Uses population that bought the product as base, % of people buying snickers were 18-24. 34% of adults with a savings acct have graduated college TERM 93
Product
summary
DEFINITION 93 Shows who bought each kind of product TERM 94
Share of users
DEFINITION 94 Of everyone who bought one of these types of the product, what % bought this type TERM 95
Share of volume
DEFINITION 95 Of all the product sold, how much was that brand
Volume/users index
Gives you an idea of amount of consumption relative to users TERM 97
Peretto principle
DEFINITION 97 20% of users account for 80% of volume
