Download Reliability, Validity, and Internal Validity in Research: Nursing 378 Exam Q&A and more Exams Nursing in PDF only on Docsity!
NURSING 378 FINAL EXAM QUESTIONS
WITH COMPLETE ANSWERS 2025
LATEST UPDATED AND VERIFIED.
Reliability - Solution The instrument consistently measures a given trait with precision The accuracy with which an instrument measures the target attribute Item-total correlation - Solution Stability among individuals Inter-rater reliability - Solution Stability between raters Test-retest - Solution Stability over time reliability coefficient. - Solution Can range from .00 to 1.00. Coefficients below .70 are considered unsatisfactory. Coefficients of .80 or higher are desirable. Stability Internal consistency Equivalence - Solution What are the three aspects of reliability? Stability - Solution The extent to which scores are similar on two separate administrations of an instrument Evaluated by test-retest reliability Test-retest reliability - Solution Requires participants to complete the same instrument on two occasions Appropriate for relatively enduring attributes Cohen's kappa 0. Internal Consistency - Solution The extent to which all the items on an instrument are measuring the same unitary attribute Evaluated by administering instrument on one occasion Appropriate for most multi-item instruments The most widely used approach to assessing reliability Assessed by computing coefficient alpha (Cronbach's alpha)
Alphas ≥.80 are highly desirable. Equivalence - Solution The degree of similarity between alternative forms of an instrument or between multiple raters/observers using an instrument Most relevant for structured observations Assessed by comparing agreement between observations or ratings of two or more observers (interobserver/interrater reliability) Low - Solution _____ reliability can undermine adequate testing of hypotheses. Procedure used to test them - Solution Reliability estimates vary depending on _________ Lower - Solution Reliability is ________ in homogeneous than heterogeneous samples Lower - Solution Reliability is ________ in shorter than longer multi-item scales. Gold standard - Solution Chronbach's Inter-rater Test-retest Internal Validity - Solution The level of confidence that an experimental treatment or condition made a difference and that rival explanations were systematically ruled out through study design and control. The ability of an instrument to consistently measure what it is suppose to measure Validity - Solution The degree to which an instrument measures what it is supposed to measure Measurement of the "right" thing Needs to be done on multiple populations, settings and situations Correlation of co-efficent used to report: 0.5 or higher is strong but 0.2-0. maybe acceptable Conditions Necessary for Causality - Solution Changes in the presumed cause must be related to changes in the presumed effect The presumed cause must occur before the presumed effect
Controlling external factors - Solution Achieving constancy of conditions Control over environment, setting, time Control over intervention via a formal protocol Controlling intrinsic factors - Solution Control over subject characteristics Methods of Controlling Intrinsic Factors - Solution Randomization Subjects as own controls (crossover design) Homogeneity (restricting sample) Matching Statistical control (e.g., analysis of covariance) Type I Error - Solution Rejection of the true Null HO You think the intervention worked - when it did not work Usually an extraneous variable that causes a _____________ Usually a design flaw Called alph (p value) - maximum level of error allowed is 0.05 (5 % chance or 5/100 chances) or 0.01 (only 1% chance - 1/100 chance of being wrong) for some treatments. Type II Error - Solution Accepting a false Null HO Stating there is "no difference" when there is a difference The treatment worked but you failed to find it significant. Also called beta Most often occurs because of insufficient power - sample size 1-power = probability of Type II error. 90% power or 0.9 power = 10% chance of Type II error. Type 1, type 1, type 2 - Solution What type of error is considered more serious due to thinking a treatment is working when it is not? When increasing control of ______ we decese control of _____? Factors that Jeopardize Internal Validity - Solution History Maturation Testing Instrumentation Treatment effects Selection effects Attrition
Temporal Ambiguity - Solution In RCTs the independent variable is manipulated and then outcome measured - best control for ______
In correlational studies may be unclear which variable occurred first! Selection - Solution There is a difference between the experimental and control groups d/t the _____________ of the subjects Biggest problem for non-experimental designs. Problem most often in "nonequivalent groups designs/case control designs Randomization or match assignment helps control for this "threat" History - Solution Events that happen at the time of data collection/intervention - like Katrina, 9/11, changed taped on the unit during and IV dsg. Study Co-intervention bias Most likely longitudinal studies or the 1 group before/after design: Maturation - Solution Subjects change- they age or get smarter or get sicker Important for longitudinal studies and especially those with children Control - Solution Use Statistics - ANCOVA Match subjects by age/level of illness Attrition (Mortality) - Solution Loss of subjects from the study Most concern in longitudinal studies Informed subjects about time commitment Screening subjects Make data collection convenient to subjects - Solution How can attrition be controlled? Testing - Solution Subjects get better at the test Pre-post test concern One group before/after design Instrumentation - Solution If data collection instrument changes during data collection or change pre to post test Two raters
Researcher may keep track of those that refuse Time - Solution Length of time for a treatment effect to become evident. Need to allow enough time for the test results to manifest Variable that may change over time - winter vs summer, morning vs night History - Solution Results must be considered in context of time period. May not be able generalized to the future Testing on a "special" day.... Can we generalize to all 365 days of the year? Novelty - Solution Subject response may just be b/c it is new or unusual Experimenter effects - Solution Researcher - effects different if done by a different person? Different sample joined b/c of person recruiting so not "true" to real world? Hawthorne effect (could effect Internal & external validity - Solution subjects respond differently b/c in study - would not respond this way in everyday life Three ways to deal with Threats - Solution Eliminate threat Control the threat Account for the threat Eliminate the threat - Solution Researcher is the threat - might designate data collection to an assistant Control the threat - Solution Distribute it across groups. Make certain some subjects with the threat are in each group Account for the threat - Solution Note it in "limitations" Statistical Analysis to Strengthen Validity - Solution Determine probability of error
- Type I
- Type II Calculating and reporting tests of effect size Ensuring data meet the fundamental assumptions of the statistical test
Internal validity - Solution provides reassurance that an intervention worked and no other causes were responsible for the outcome External validity - Solution gives the nurse confidence in generalizing the results to other populations When reading for validity.... - Solution No labeled section No ok or horrible number of threats Each threat is evaluated for the potential effect on your ability to "trust" the results MUST HAVE INTERNAL VALIDITY TO HAVE EXTERNAL VALIDITY! Check out the method, sampling procedures, measurement, and statistical analysis Check what is noted as a "limitation" or "weakness" Replication, Meta-analysis, Systematic reviews compensate for threats to internal validity Descriptive statistics - Solution Used to describe and synthesize data Statistics that describes the data in a form that is readily understandable; convert a collection of data into a picture of the information that has some meaning for the consumer Inferential statistics - Solution Used to make inferences about the population based on sample data Allows the investigator to decide whether the outcome of the study is a result of factors planned within the design of the study or determined by chance; allows for inferring to a larger population General Rules for Quantitative Analysis - Solution Statistical tests are selected a priori The acceptable level of significance is also selected before analysis begins Run all of the identified tests Report all of the tests that were run Selective reporting is a source of bias Frequency Distributions - Solution A systematic arrangement of numeric values on a variable from lowest to highest, and a count of the number of times (and/or percentage) each value was obtained can be described in terms of: Shape
Range - Solution highest value minus lowest value. Standard deviation (SD) - Solution average deviation of scores in a distribution Most frequently used statistic for measuring the degree of variability in a set of scores Uses every value in a distribution Summary of the average amount of deviation of values from the mean Tells how variable scores in a distribution are Roughly 3 SD above and below the mean 68% fall within 1 SD above and below the mean 95% fall within 2 SD from the mean About 2% at each extreme - more than 2 SDs from the mean Variance - Solution Reflects the amount of variation in a data set Measure of dispersion, where the large the variance, the larger the dispersion of scores. Variance is calculated as one of the steps in determining standard deviation. Large values (close to 1.0) reflect greater _________ in data set Small values (close to zero) reflect less _________ in data set Bivariate Descriptive Statistics - Solution Contingency tables - frequency distribution in which the frequencies of two variables are cross-tabulated Usually used with nominal data or ordinal data that have few levels or ranks Correlation Coefficients -1.00 to + 1.00 - Solution Used for describing the relationship between two variables To what extent are two variables related to each other? The greater the absolute value of the coefficient, the stronger the relationship: Correlation - Solution Pearson's r is both a descriptive and an inferential statistic. Tests that the relationship between two variables is not zero. +1.00 - Solution Perfect relationship Positive relationship - Solution means increments in one variable are associated with increments in the second: .00-+1.
Negative relationship - Solution two variables are inversely related, increments in one variable area associated with decrements in the second; .00 to -1. Statistical Inference - Solution Questions about reliability answered by setting confidence limits Questions about probability answered by hypothesis testing Conclusions concerned with probability of drawing an erroneous conclusion interval or ratio measures - Solution Pearson's product-moment correlation coefficient (r) most commonly uses ... ordinal measures - Solution Spearman's rank-order correlation (r2) uses ... Errors in Summarizing Data - Solution Use of an inappropriate statistic (ie: mean never done for nominal data - gender, race, etc.) Demographic data is usually frequency or percentages Common Error - Over Interpretation! - Solution Most common with correlation - they want to say one variable "caused" the other. Only an association - not able to draw causal conclusions Confidence Interval - Solution Two numerical values defining an interval that we believe, with an identified level of confidence, actually includes the estimated population parameter Interval estimation - Solution A range of values within which a population value probably lies Involves computing a confidence interval (CI). confidence interval - Solution indicate the upper and lower confidence limits and the probability that the population value is between those limits. Hypothesis Testing - Solution Statistical test of significance Between a sample and a known population Between two samples Between two variables in a sample Can be used to test differences between: Means Proportions Variances
Parametric Statistics - Solution Random samples from defined population Dependent variable measured at interval or ratio level Estimation of at least one variable More powerful Make assumptions about population from which sample was drawn Nonparametric Statistics - Solution Do not require the assumptions of parametric test Normal distribution not required Measures data on nominal or ordinal level Less powerful than parametric Parametric- see handout - Solution t- test for independent groups t-test for dependent groups Analysis of variance (ANOVA) Repeated measures ANOVA Pearson's r Nonparametric tests-see - Solution Chi-squared Mann-Whitney U-Test Kruskal-Wallas test Wilcoxon signed ranks test Friedman test Phi coefficient Spearman's rank-order correlation r Multivariate Statistics - Solution Statistical procedures for analyzing relationships among 3 or more variables Two commonly used procedures in nursing research: Multiple regression Analysis of covariance (ANCOVA) SPSS - Solution most often used in Nursing Research. Stats for Bio Sciences SAS - Solution more purely mathematical Most Common Reported Statistics - Solution Descriptive statistics about sample and variables
Analysis of group equivalency Statistics about the role of error Statistics to evaluate magnitude of effect Statistics to determine confidence level Nominal - Solution Categorical data / labels / no mathematical properties Ordinal - Solution Categorical data that are ranked Interval - Solution Data that are ranked with equal intervals Ratio Data - Solution Interval level data that have a true zero Level of measurement - Solution A variable's _______________ determines what mathematic operations can be performed in a statistical analysis. Nominal Level of Measurement - Solution Lowest level of measurement Assigning numbers to categories Number assignment "labels" category but does not indicate order or magnitude. Example Gender has 2 categories 0 = Male 1 = Female One category is not higher or lower than another Number assigned is for labeling purposes only Ordinal Level of Measurement - Solution Second level of measurement Assigns numbers to categories but there is a rank order HOWEVER the exact differences between the categories cannot be determined. Example: Anxiety - Low (score < 20), Med (21-50), High (> 50) Higher score more anxiety Level of Education - 1 -4 years, higher number more education Interval Level of Measurement - Solution Rank order with specified distance between measures "real" numbers on a scale Provides more depth to data analysis. Can calculate a mean score
