extent to which a method measures what it is supposed to measure; truthfulness or accuracy of a measure

The degree to which variability (individual differences) in observed measures reflect individual differences in the characteristic or construct we want to measure.

Three types:

1. Face validity: the extent to which a measure 'appears' to measure what it is supposed to measure; this involves the judgment of the researcher and participants and is not statistical; a measure has face validity if people think it does; just because a measure has face validity does not mean it is a valid measure (and measures lacking face validity can be valid measures)

2. construct validity: we often want to measure hypothetical constructs that can't be directly observed (for example, intelligence) but are inferred from empirical evidence; construct validity is assessed by studying the relationships between the measure of a contruct and scores on measures of other constructs; we assess construct validity by seeing whether a particular measure relates as it should to other measures.

3. criterion validity: the extent to which a measure (1) distinguishes participants on the basis of a particular behavioral criterion or (2) matches a 'gold standard' referent. Example: the SAT is valid to the extent that it distinguishes between students that do well in college versus those that do not; a valid measure of marital conflicts should correlate with behavioral observations (eg, number of fights); a food frequency questionnaire is a valid measure of fruit and vegetable consumption if it is highly correlated with serum carotenoid levels.

Sensitivity and Specificity can be used to assess the validity of a binary (dichotomous) measure.

A highly sensitive measure will yield few false negatives (sensitivity = TP/(TP+FN))

A highly specific measure will yield few fals positives (specificity = TN/(TN+FP))

A measure's sensitivity and specificity affect the relative risk calculation (please see slides and do the example).

Remember:

Sensitivity: proportion of people with a characteristic who are correctly identified as such by the measurement

Specificity: proportion of people without a characteristic who are correctly identified as such by the measurement.

SEE SLIDES for more on how sensitivity and specificity affect relative risk. Example given: A decre

-test-rest reliability: consistency of participant's responses over time; assumes the characteristic being measured is stable over time and won't change between test and retest.

-interrater reliability: if a measurement involves behavioral ratings by an observer/rater, we would expect consistency among raters for a reliable measure; best to use at least 2 independent raters who are blind the ratings of other observers; precise operational definitions and well-trained observers improve the interrater reliability.

-internal consistency reliability: assesses the degree of consistency among the items in a scale or the different observations used to derive a score; it's important that all the items (or observations) are measuring the same construct.

The analyst's goal is to isolate between-subject (that is, biological, or real) variability, while eliminating less interesting sources of measurement variability.

The ability to detect between-subject differences depends on reliability and validity.

If variability due to extraneous sources is greater than between-subject variability, it may be impossible to detect true biologic differences ("signal to noise" ratio).

Find an objective measure where possible; remove human judgment

Enhance measurement validity: reduce respondent burden, provide memory aids or diaries, use randomized response technique, bogus pipeline (?), use more than one observer or rater.

Make replicate measurements

Standardization of duplicate equipment, labs, etc.

1947: EL Wynder, in the AJE 1997,remembers, "generally the medical profession in those days did not think seriuosly about smoking as a potential cause of major diseases."

1948: EL Wynder, in the AJE 1997, says that the medical director of American Cancer Society in 1948 said, "On smoking and lung cancer, my mind is closed." But ACS funded Wynder's case-control study on smoking and lung cancer.

1950: Publication of 3 key case-control studyies linking smoking to lung cancer (Levin, Wynder & Graham, and Doll & Hill).

Doll & Hill: "Smoking & Carcinoma of the Lung, Preliminary Report" (British Medical Journal). Cases (hospitalized lung-carcinoma patients) and controls (hospitalized non-cancer patients) are controlled well for age and sex; controlled for social class; and less well controlled for place of residence. Smoker status assessed though two interviews. Interpretation of results / conclusion: "To summarize, it is not reasonable to attritube the results to any special selection of cases or to bias in recording. In other words, it must be concluded that there is a real association between carcinoma of the lung and smoking. ... As to the nature of the carcinogen we have no evidence."

1956: Doll & Hill: "Lung Cancer and Other Causes of Death in Relation to Smoking: A 2nd Report on the Mortality of British Doctors" (British Medical Journal). Survey of British physicians (70% response rate) from 1951-1956 assessed death rates per year in men 35+ in relation to most recent amount smoked. Found biological gradient for lung cancer in smokers based on amount smoked.

1954-56: Doll & Hill publish results of their prospective cohort study of British doctors showing that smoking is associated with increased risk of lung cancer and heart disease

1957: US NCI, US NHLBI, AHA and ACS ad hoc study group concludes that the "sum total of scientific evidence establishes beyond reasonable doubt that cigarette smoking is a causative factor" in lung cancer.

1958: RA Fisher (1890-1962) remained skeptical of the tobacco-lung cancer link and proposed the 'constitutional hypothesis' in his 1958 article in Nature on smoking habit in twin pairs: "both characteristics [smoking habit and lung cancer] might be largely influenced by a common cause, in this case, individual genotype."

1960: WHO concludes that smoking causes lung cancer.

1962: Wynder gets a new boss at Memorial Sloan-Kettering Cancer Center. Dr. Frank Horsfall, while smoking a cigarette, informs Wynder that he will censor all future publications from Wynder on smoking and lung cancer. Wynder responded (eventually) by leaving to form the American Health Foundation.

1964: US Surgeon General's report concludes that prolonged smoking is a causative factor in etiology of lung cancer.

1988: US Surgeon General concludes that smoking is addictive.

1994: Heads of 7 leading tobacco companies appear before US Congress and deny under oath that nicotine is addictive.

1998: master tobacco settlement between major US companies and states.

Methodological advances in epidemiologic research which can be attributed to work on smoking and lung cancer

Case-control study design, analysis, matching

Methods to assess etiologic relationships with observational data

Bradford Hill guidelines for establishing causality

Understanding of confounding, effect modification, and how to control for the effects of both.

Statistical analysis approaches to observational evidence.