Download RELIABILITY AND VALIDITY and more Assignments Research Methodology in PDF only on Docsity!
RELIABILITY AND VALIDITY OF TOOLS
INTRODUCTION
Quantitative studies derive data through the measurement of variables. Measurement involves the assignment of numbers to represent the amount of an attribute present in an object or person, using a specified set of rules. RELIABILITY OF TOOLS The reliability of a quantitative instrument is a major criterion for assessing its quality and adequacy. DEFINITION RELIABILITY āis the degree of consistency or dependability with which an instrument measures an attributeā If a scale weighed a person at 120 pounds one minute and 150 pounds the next, we would consider it unreliable. The less variation an instrument produces in repeated measurements, the higher its reliability. Thus reliability can be equated with a measureās stability, consistency, or dependability. Reliability also concerns a measureās accuracy. An instrument is reliable to the extent that its measures reflect true scores- that is, to the extent measurement errors, are absent from obtained scores. A reliable measure maximizes the true score component and minimizes the error component. These two ways of explaining reliability( consistency and accuracy) are not so different as they might appear. Errors of measurement that impinge on an instruments accuracy also affect its consistency. The reliability of an instrument can be assessed in various ways, and the appropriate method depends on the nature of its instrument and on the aspect of reliability of greatest concern. Three key aspects are stability, internal consistency and equivalence. ļ¶ STABILITY The stability of an instrument is the extent to which similar results are obtained on two separate occasions. The reliability estimate focuses on the instrumentās susceptibility to extraneous influences over time, such as participant fatigue. Assessments of an instrumentās stability involve procedures that evaluate test-retest
reliability. Researchers administer the same measure to a sample twice and then compare the scores. The comparison is performed objectively by computing a reliability co-efficient, which is an index of the magnitude of the testās reliability. To explain reliability co-efficients we must briefly discuss a statistic called correlational co-efficient. For example, is there a relationship between patientsās gastric acidity levels and level of stress experienced? The correlational co-efficient is a tool for quantitatively describing the magnitude and direction of a relationship between two variables. It is more important to understand how to read a correlation coefficient. Two variables that are obviously related are peopleās height and weight. Tall people tent to be heavier than short people. We would say that there was a perfect relationship if the tallest person in an population were the heaviest, the second tallest were the second heaviest and so forth. Co-rrelational co-efficients summarize how perfect relationships are. The possible values for a correlation co-efficients range from -1.00 through 0.00 to +1.00. if the height and weight were perfectly correlated, the correlation co-efficient expressing this relationship would be 1.00. Because the relationship exists but is not perfect, the correlation co-efficient is typically in the vicinity of 0,50 or 0.60. the relationship between height and weight can be described as a positive relationship because increases in height tend to be associated with increases in weight. When two variables are totally unrelated, he correlation equals zero. One might expect that womenās dress sizes are unrelated to their intelligence. Large women are as likely to perform well on I.Q. tests as small women. The correlation co-efficient summarizing such as a relationship would presumably be in the vicinity of 0.00. correlation co-efficients running from 0.00 to -1.00 express inverse or negative relationships. When two variables are inversely related, increases in one variable are associated with decreases in the second variable. Suppose that there is an inverse relationship between peopleās age and the amount of sleep they get. This means that , on average, the older the person, the fewer the hours of sleep. If the relationship were perfect, the correlation coefficient would be -1.00. In actuality, the relationship between age and sleep is probably modest in the vicinity of -0.15 or -0.20. a correlation coefficient of this magnitude describes a weak
a higher consistency. Coefficient alpha is usually used as an index of internal consistency to estimate the extent to which different subparts of an instrument are reliably measuring the critical attribute. ļ¶ EQUIVALENCE Equivalence in the context of reliability assessment, primarily concerns the degree to which two or more independent observers or coders agree about the scoring on an instrument. If there is a high level of agreement, then the assumption is that measurement errors have been minimized. Nurse researchers are especially likely to use this approach with observational measures, although it can be used in other applications. The reliability of ratings and classification can be enhanced by careful training and the specifications of clearly defined, non-overlapping categories. Interrater ( or inter observer) reliability can be assessed using a variety of approaches which can be categorized as consensus, consistency and measurement approaches. Many of the indexes of interrater reliability used by nurse researchers are of the consensus type which are based on the assumption that the goal is to have observers share a common interpretation of the construct under observation and to reach consensus( exact agreement ). Consensus measures of interrater reliability for observational coding involve having two or more trained observers watching an event simultaneously and independently recording data according to the instruments instructions. The data can then be used to compute an index of agreement between observers. INTERPRETATION OF RELIABILITY COEFFICIENT Reliability coefficients are important indicators of an instrumentās quality unreliable measures reduce statistical power and hence effect statistical conclusion validity. If data fail to support a hypothesis one possibility is that the instruments reliability were unreliable not necessarily that the expected relationships do not exist, knowledge about an instruments reliability thus is critical in interpreting research results, especially if research hypothesis are not supported. Reliability coefficients have a special interpretation that should be briefly explained. This interpretation relates to the earlier discussion of decomposing observed scores into error components and true components. Suppose we administer a scale that measures hope to 50 cancer patients it would be expected that the scores vary from one person to
another, that is some people would be more hopeful than others. Some variability in scores is true variability, reflecting real individual differences in hopefulness; some variability is however is error, thus, V 0 = VT + VE Where Vo = observed total variability in scores. VT = true variability VE = variability owing to errors A reliability coefficient is directly associated with this equation. Reliability is the proportion of true variability to the total obtained variability or r = VT/VO. FACTORS AFFECTING RELIABILITY Various things affect an instruments reliability and these factors are useful to keep in mind in developing an instrument or selecting one for use. First the reliability scales is partly a function of their length ( i.e. number of items). To improve reliability, more items tapping the same concept should be added. Items that have no discriminating power should however be removed. With observational scales, reliability can usually be improved by greater precision in defining categories, or greater clarity in explaining the underlying dimension for rating scales. The most effective means of enhancing reliability in observational studies, however, is thorough training of observers. The reliability of an instrument is related in part to the heterogeneity of the sample with which it is used. The more homogenous the sample, the lower the reliability co efficient will be. This is because instruments are designed to measure differences among those being measured. If the sample is homogenous then it is more difficult for the instrument to discriminate reliably among those who possess varying degrees of the attribute being measured. For example, a depression scale will be less reliable when it is used with a general population. Choosing an instrument previously demonstrated to be reliable is no guarantee of its high quality in a new study. An instrumentās reliability is not a fixed entity. The reliability of an instrument is a property nit of the instrument but rather of the instrument when administered to a certain sample under certain conditions. A scale that reliably measures dependence in hospitalized adults may be unreliable with nursing home residens. This means that in selecting an instrument, it is important to know the characteristics of the
CONTENT VALIDITY
Content validity concerns the degree to which an instrument has an appropriate sample of items for the construct being measured and adequately covers the construct domain. Content validity is relevant for both affective measures (i.e. measures relating to feelings, emotions and psychological traits.) and cognitive measures. An instrumentās content validity is necessarily based on judgement. There are no completely objective methods of ensuring the adequate content coverage of an instrument. There are no completely objective methods of ensuring the adequate content coverage of an instrument. There are various approaches to assessing content validity using an expert panel, but nurse researchers have been in the forefront of developing an approach that involves the calculation of a content validity index (CVI). The experts are asked to evaluate individual items on the new measure as well as the overall instrument. At the item level, a common procedure is to have experts rate items on a 4- point scale of relevance, the scale that seems to be used most often is as follows;- 1= not relevant, 2= somewhat relevant, 3= quite relevant, 4= highly relevant. Then for each item, the item CVI is computed as the number of raters giving a rating of either 3 or 4, divided by the number of experts ā i.e. the proportion in agreement about relevance. For example, an item rated as āquiteā or āhighlyā relevant by 4 out of 5 judges would have an I-CVI of 0.80, which is considered an acceptable value. CRITERION RELATED VALIDITY Establishing criterion related validity involves determining the relationship between an instrument and an external criterion. The instrument is said to be valid if its scores correlate highly with scores on the criterion. For example, if a measure of attitudes toward premarital sex correlates highly with subsequent loss of virginity in a sample of teenagers, then the attitude scale would have had good validity. One requirement of this approach is the availability of a reliable and valid criterion with which measures on the instrument can be compared. This is unfortunately, seldom easy. Criterion- related validity is most appropriate when there is a concrete, reliable criterion. A distinction is sometimes made between two types of criterion related validity.
- Predictive validity- refers to the adequacy of an instrument in differentiating between peopleās performance on some future criterion.
- Concurrent validity- refers to an instrumentās ability to distinguish individuals who differ on a present criterion. The difference between predictive and concurrent validity, then , is the difference in the timing of obtaining measurements on a criterion. CONSTRUCT VALIDITY Construct validity is a key criterion for assessing the quality of a study, and construct validity has most often been addressed in terms of measurement issues. Construct validity concerns inferences from the particulars of the study to the higher-order constructs of interest. Validating an instrument in terms of construct validity is a challenging task. Construct validity is essentially a hypothesis testing endeavor, typically linked to a theoretical perspective regarding the construct. There are a number of ways to gather evidence about the construct validity of a scale, most of which are discussed below;- ļ¶ KNOWN GROUPS One construct validation approach is the known-groups technique. In this procedure, the instrument is administered to groups hypothesized to differ on the critical attribute because of some known characteristic. For example, in validating a measure of fear of the labor experience, we could contrast the scores of primiparas and multiparas. We would expect that women who had never given birth would be more anxious than women who had done so, and so we might question the instrumentās validity if such differences did not emerge. We would not necessarily expect large differences; some primiparas would feel less anxiety and some multiparas would express some fears. On the whole, however we would anticipate differences in average group. ļ¶ HYPOTHESIZED RELATIONSHIPS A similar method of construct validation involves testing hypothesized relationships, often on the basis of theory. This is really a variant of the known groups approach which involves hypothesis about the relationship between the measure of the construct and a variable representing group membership. A researcher might reason as follows;- ļ· According to theory, construct X is positively related to construct Y.
measuring what it is supposed to be measuring, the more confidence researches will have in its validity. CONCLUSION Reliability and validity are the two most important criteria for evaluating quantitative instruments. High reliability and validity of measuring tools are necessary for high-quality-quantitative research.
PROCEDURE FOR DATA COLLECTION
General procedure Whichever data collection approach is chosen the following procedure must be used: o Search the literature to find if similar data collection tool has been used or if guidance in developing a new one exists. o Try a draft with colleagues and make any changes o Carry out a pilot study, which will include data analysis and make any changes o Always ask throughout his process ā āWill it answer the research questions?ā, āAm I happy about how the data will be analysed?ā āHow valid and reliable will my data be?ā. THE DATA COLLECTION INVOVLES TWO STEPS ļ· Identify the parameters for data collection: researchers should consider four parameters suggested by Miles and Huberman : ļ¼ The setting:- where the research will take place ļ¼ The actors:- who will be observed or interviewed ļ¼ The events:- what the actors will be observed doing or interviewed about
ļ¼ The process:- the evolving nature of events undertaken by the actors within the setting. ļ· Identify the type or types to be collected and provide a rationale for data collection like observation, interviews etc.
INDEX SL NO (^) CONTENT PAGE NO 1 2 3 4 5 6 Introduction Reliability of tools ļ· Definition ļ· Key aspects ļ· Interpretation of reliability co-effeicients ļ· Factors affecting reliability Validity ļ· Definition ļ· Face validity ļ· Content validity ļ· Criterion related validity ļ· Construct validity ļ· Interpretation of validity. Conclusion Procedure for data collection Bibiliography
SMT. NAGARATHNAMMA COLLEGE OF NURSING
ASSIGNMENT IN NURSING
RESEARCH
ON
RELIABILITY AND VALIDITY OF TOOLS
Submitted to:
Mrs. GEETHA .C
H.O.D. CHN(N)
Smt nagarathnamma college of
Nursing. Submitted by:
Ms. Linda Rose.J
M.Sc. (N), 1
ST
year.
Smt. Nagarathnamma
College of nursing.
Submitted on:13/01/
