Internal Validity

The ability to control the extraneous variables on the dependent variable.

Anytime there are alternative explanations for the effect of the independent variable on the dependent variable, internal validity is threatened.

External Validity

The ability to generalize the study results to other groups and settings beyond those in the current experiment.

Research that can be generalized will have more practical value.

Threats to Internal Validity

Extraneous factors that allow for alternative explanations as to what caused the given effect on the dependent variable.

History Effect

A threat to internal validity when events occur between the pretest and posttest of an experiment that could affect participants in such as way as to impact the dependent variable (stress, illness, etc).

What can a researcher do? Use a control group that has not been effected or mildly effected by the even.

Control Group

Experimental Group

Maturation Effect

Occurs when there are changes seen in subjects because of the time that has elapsed since the study began and which may not be the result of of any program effects. Most common in children being studied.

What can a researcher do? Use a control group.

Testing

In many types of research it is necessary to pretest participants to establish a baseline of the dependent variable. The pretest is usually compared to the posttest measurement to determine the effectiveness of a treatment. However, by being tested, participants may become "test wise". Any differences seen from the basline to the posttest may be form testing effects.

What can a researcher do? Use a control group that does not receive any pretesting.

Test Wise

Instrumentation

Instrumentation used to collect data can cause threats to internal validity where measurements are not accurate or procedures are not standardized.

What can a researcher do? Use a well designed instrument that accurately measures what it is supposed to measure.

Selection Bias

Anytime individuals are selected in a non-random manner, selection bias may be present. Example: A study on a new health site, it is probably that those who choose to participate are interested in health.

What can a researcher do? Although this cannot be completely countered, recruiting volunteers and then randomly assigning them is better than allowing them to self-select a group. Also, matching them on selected characteristics and then randomly assigning them can help. Finally, pretesting groups on measures of the dependent variable to make sure there are no pretreatment differences between groups can help.

Selection Maturation Effect

A combination of selection bias and maturation effect. This occurs when using intact groups that vary in their maturation level.

What can a researcher do? Pretesting and/or prescreening groups on maturity levels.

Statistical Regression

Occurs when participants are selected on the basis of their extremely high or low scores. If these same participants were administered the same instrument again, the tendency would be for those who scored extremely high on the first test to score lower on the second test and vice versa.

What can a researcher do? The study could be designed to follow a random sample of individuals representing a full range of scores rather than placing participants in high and low groups based on one testing.

Mortality/Attrition

It is common to lose participants over the course of a study. In longitudinal studies, some may die; in others, some will drop out or move without leaving contact information. This creates a "mortality effect". Loss of subjects hurts the validity of comparisons.

What can a researcher do? Provide incentive for participants.

Hawthorne Effect

When participants' attitudes toward being involved in a study affect the way they behave.

What can a researcher do? Try to provide the control group with some special treatment that is comparable to the experimental group but would not have direct impact on the independent variable. Also, keep participants from knowing that htye are taking part in a study or being observed.

Placebo Effect

Effects caused by participants' expectations rather than the actual treatment.

What can a researcher do? Use a blind or double-blind study. Also, ensure that those in both the control and experimental groups receive the same information so that both groups have similar expectations. Lastly, give both groups as little information as possible to evade expectations.

Diffusion & Treatment

The Diffusion effect occurs when the treatment being applied to one group spills over or contaminates another group.

What can a researcher do? Use control/experimental groups that live separately from each other. Urge participants to keep personal effects of the treatment confidential until the end.

Location

Location effect occurs when there are differences in the locations where interventions take place.

What can a researcher do? Make the locations the same for all participants. If that's not possible, the researcher should try and minimize location differences that affect the dependent variable.

Implementation

Incolges the individual or individuals responsible for implementing the experimental treatment and the possibility that they may inadvertently introduce inequality or bias into the study.

What can a researcher do? Make sure all persons responsible for implementing the program are equally trained and competent, and following standardized protocol. Another way is to have somone other than the program developer present the program. Lastly, internal validity can be controlled by having a neutral observer watch the presenter.

Threats to Internal Validity

Selection Treatment Interaction

Concerns the ability of the researcher to generalize the results of a study beyond the groups involved in the study.

What can a researcher do? Random selection from a population to form a study population.

Setting Treatment Interaction

Concerns the extent to which the environmental conditions or setting under which an experimental study was conducted can be duplicated in other settings.

What can a researcher do? Consider differences in backgrounds of different populations as well as the settings they are given the study.

History Treatment Interaction

This develops when the researcher tries to generalize findings to past and future situations. Some experiements may be time sensitive and not produce similar results if conducted earlier or later.

What can a researcher do? Take into consideration effects of timing, and adjust the study accordingly.

Increasing Internal & External Validity

Maximize Internal Validity by using the Analysis of Covariance (ANCOVA). This analysis equalizes many initial differences that might exist between groups.

Maximize External Validity by carefully considering what groups can one legitimately generalize to. Also, always duplicate a setting and historical factors as nearly as possible when replicating a study with a different population.

REMEMBER

External Validity CANNOT exist without Internal Validity!

Researchers should always be concerned about ensuring Internal Validity before trying to establish External Validity!