Validity
Accuracy of a measure
Validity is defined by COSMIN as “the degree to which an instrument truly measures the construct(s) it purports to measure”1
| Validity | Reliability |
|---|---|
| Deals with the accuracy ofinferences made from measurements2 | Deals with the reproducibility of measurements themselves2 |
| Concerns the relationship between the measurement and the entity being measured2 | Is a property ofthe measurement (and the person performing it) |
| Requires independent knowledge ofthe “true” value of the entity being measured2 | Is not dependent on the “true” value ofthe entity being measured2 |
| Presupposes a certain degree of reliability2 | Does not presuppose validity2 |
| Is undermined by systematic error2 | Is undermined by random error2 |
| Liable, if lacking, to distort or bias relationships among variables2 | Liable, if lacking, to obscure relationships among variables2 |
3 Types
There are 3 validity components in both COSMIN and Polt-Yang Taxonomies1
Content/Face Validity
Content Validity
Content validity is one of the 3 main types of validity and refers to “the degree to which the content of an instrument adequately reflects the construct being measured”1. Content validity represents an early method to enhance construct validity of an instrument1.
Claims about the validity of an instrument should never be based exclusively on evidence of adequate content validity1
Face Validity
Face Validity is often considered a subdomain of content validity and refers to “the extent to which an instrument looks as though it is a measure of the target construct”1. Face validity is a purely qualitative and subjective judgement made by an examiner.
Criterion Validity
Criterion validity is one of three main components of validity and explains how well the test in question relates to the “gold standard” of the same construct. Most patient-reported outcomes have no “gold standard” and thus researchers should determine convergent validity instead of criterion validity1.
Criterion validity has 2 forms:1
- Concurrent validity
- Predictive Validity
The key feature of a criterion validity approach is that there must be a ‘‘gold standard’’ criterion against which scores on the focal measure can be assessed.
Concurrent Validity
Concurrent Validity is a type of criterion validity that tests whether a measure is consistent with the “gold standard,” measured at the same time1.
Predictive Validity
Predictive Validity is a type of criterion validity that tests whether a measure can predict the outcome of the gold standard measured at a future point in time1
Construct Validity
Construct Validity is one of the 3 main types of validity and refers to the “degree to which evidence about a measure’s scores supports the inference that the construct has been appropriately represented”1.
Hypothesis-testing Validity
Hypothesis-testing validity is a subdomain of construct validity and refers to1
Hypothesis-testing validity can take many forms:
Convergent Validity
Convergent Validity is a form of hypothesis-testing validity and is applied in the absence of a gold standard to test “the correlation between scores on the focal measure and scores on a measure of a construct with which conceptual convergence is expected”1
Divergent Validity
Divergent (Discriminant) Validity is a form of hypothesis-testing validity that tests the hypothesis that the outcome measure does not measure any other constructs other than the one intended1
Known Groups Validity
Known Groups (Discriminative) Validity is a form of hypothesis-testing validity that tests the hypothesis that “the degree to which a measure can discriminate between groups known to differ with regard to the focal construct”1.
Structural Validity
Structural validity is a subdomain of construct validity that uses factor analysis to test if a measure captures the hypothesized dimensionality of a construct1.
Cross-Cultural Validity
Cross-cultural validity is a subdomain of construct validity and refers to1
Cross-cultural validity is relevant for the validation of a cultural or linguistic adaptation of an instrument1.
“Concerns the extent to which a translated or adapted measure is equivalent to the original”
“Cross-cultural validity, the third type of construct validity, concerns the extent to which evidence supports the inference that the original and a translated or culturally adapted scale are equivalent. In the sample of 105 nursing studies, a full 36 (34.3%) of them involved efforts to assess the cross-cultural validity of a translated scale.”
“In summary, nurse researchers could strengthen their validity claims in instrument studies by testing thoughtful, theory-based hypotheses about the extent to which the measure yields scores that ‘‘behave’’ as predicted in relation to other constructs—or by identifying an appropriate gold standard for a criterion validation. Factor analysis alone as a construct validity strategy does not directly answer the central validity question: Does the scale measure the construct it purports to measure? Exploratory factor analysis is an important tool for finalizing or refining a multi-dimensional instrument, but confirmatory factor analysis should be the method of choice for structural validation.”
Internal & External Validity
There are 2 broad types of validity: External and Internal validity2
Internal Validity
Internal validity refers to the possibility that the the conclusions drawn from the results of the study accurately reflect the experiment itself2.
Includes:
- Construct validity: Does the study design correctly answer its research question?
- Statistical Conclusion validity: Were the correct statistical tests used and were they interpreted correctly?
External Validity
External validity refers to how successfuly one can apply the results of a study on a sample can be generalized to a particular population2.
Face validity is often identified with content validity since is less applicable to formal scientific testing2.
Improving Validity
“In order to improve validity, attempts must be made primarily to remove systematic error, or bias”2