Understanding Concepts, Variables, Validity, and Operationalization in Research, Study notes of Design

Download Understanding Concepts, Variables, Validity, and Operationalization in Research and more Study notes Design in PDF only on Docsity!

Concepts, Variables,

Validity and

Operationalization--

and threats to

measurement

quality

Concept

A mental image that refers to a

unified set of ideas.

 Page 79 (4.1)

Research design – operationalization of

variables

 The construction of actual, concrete measurement

techniques; the creation of “operations” that will

result in the desired measurements.

 The development or choice of specific research

procedures (operations) that will result in representing

the concepts of interest.

Operationalization

 An operational definition is a procedure for classifying,

ordering, or quantifying something

 Classifying - crowded or not crowded  Ordering - uncrowded, mildly crowded, severely crowded  Quantifying - measure crowdedness in terms of the number of residents per square kilometre

Choices to be made about operationalization

 The range of variation – how large should your categories be?

 Depends on the purpose of your study – pragmatic considerations (e.g., income)

 Variation between the extremes – how fine are the distinctions you want to make in your study?  e.g., age  Again, depends on the purpose of your study

 Single or multiple indicators of variables

 Some straightforward, such as gender  But others benefit from multiple indicators (e.g., Q3A-L)

Operationalizing Choices

 The process of creating a definition(s) for a concept

that can be observed and measured

 The development of specific research procedures that

will result in empirical observations

 Examples

 SES is defined as a combination of income and education and I will measure each by…  The development of questions (or characteristics of data in qualitative work) that will indicate a concept

Independent and Dependent Variables

 Independent variable is what is manipulated (or it is the subject or grouping variable)

 a treatment or program or cause

 ‗Factor‘

 ‗Explanatory Variable‘

 Dependent variable is what is affected by the independent variable

 effects or outcomes

 ‗Measure‘

 ‗Response Variable‘

Validity

The extent to which our measure reflects what we

think or want them to be measuring

 Internal Validity – design and measurement

concerns that reduces chances for internal errors.

 External Validity– describes our ability and intent to

generalize to subjects beyond our study sample.

Largely an issue of design and sampling.

External Validity

 External validity addresses the ability to generalize your

study to other people and other situations.

 To have strong external validity (ideally), you need a probability sample of subjects or respondents drawn using "chance methods" from a clearly defined population.

 Ideally, you will have a good sample of groups and a sample of measurements and situations.

 When you have strong external validity, you can generalize to other people and situations with confidence.

Internal Validity

 Specifically, measurement validity

 Measures are valid for a single purpose

 Four types of validity:

1. Face—as judged by others or by logic
2. Content—captures the entire meaning of the experience
3. Criterion—agrees with a validates, reliable external source:  Concurrent, agrees with a preexisting measure  Predictive, agrees with a future behavior or outcome
4. Construct—a measure is related to other measures as suggested by a theory

Reliability

 Repeated use of the measure with identical subjects

yields identical and consistent results. It is improved

by:

 Clear conceptualization

 Precise measurement

 Multiple indicators

 Pilot-testing

Reliability

 A measurement that produces consistent scores when

the object or phenomena does not change or has

experienced known change.

Validity and Reliability

Reliable, NOT valid

Valid, NOT Reliable

NOT Valid, NOT Reliable

Valid AND Reliable

NOT Valid and UNReliable

Valid and Reliable

NOT Valid but Reliable

Valid but UNReliable