Exists if the effects of the independent variable on the dependent variable are not caused by extraneous factors.

There are 13 common threats to internal validity (IV.

The ability to generalize the results of a study to other areas/groups.

There are three common threats to external validity (EV).

IV-1

History Effect

• Event that happens between the pre & post test that affects the particpation in a study.

Solution: Establish a control group that will not/has not been exposed to the event. 

IV-2

Maturation Effect

• Study subjects have changed over time: older,wiser, stronger, etc. 

Solution: Implementing an control group will eliminate the threat of maturation.

IV-3

Testing Effect

• Participants may learn how to do better on the next assessment from the pretesting.  This is known as being "test-wise".

Solution: Use a control group that doesn't receive the pretest.

IV-4

Instrumentation

• When measurements or procedures used are not consistent  or standardized.

Solution: Ensure the instrument is well designed and the questions being asked are designed to find the specific answers that are trying to be measured.

IV-5

Selection Bias

• When participants are organized/grouped in a non-random method. Often seen in field experiments with intact groups.

Solution: Randomly assign volunteers to groups. Match volunteers with similar characteristics and then randomly assign to groups. Pretesting  can also be used to increase randomization.

IV-6

Selection Maturation Effect

• When intact groups have different levels of maturation.

Solution: Group members can be prescreened or pretested to reduce this threat.

IV-7

Morality/Attrition

• Common in longitudinal studies.
• Some participants may die, be in poor health or drop out.

Solution: Create an initial group of participants that is large enough to accomodate for the loss of participants without affecting the study.

IV-8

Hawthorne Effect

• The attitudes of the individuals in a study start to affect the way they behave. These changes are not attributed to the intervention being measured.

Solution: Provide both the study and control group with special treatment/attention that does not directly impact the dependent variable.

IV-9

Placebo Effect

When members of a study group believe (their expectations) they are seeing changes but this is not due to any intervention.

Solution: Implement a blind or double blind study. Also provide groups in the study with minimal information about the study. 

IV-10

Diffusion of Treatment

• When a treatment given in a study group is shared with members of a control group.

Solution: Ensure the two groups in the study are in different locations and/or do not interact.  If this is not an option, stress with the study group the importance of not sharing the information with non-group members.

IV-11

Location

• When the location/environment conditions differ and may affect the testing conditions/results.

Solution: Ensure the testing location is the same for all participants. If this is not possible work to make the conditions as similar as possible.

IV-12

Implementation

• When multiple people are providing treatment and not doing so in the same way (training, knowledge).
• When the researcher/person providing the treatment favors one group over another.

Solution: A. ensure all people are equally trained and following established protocol.

B.  Have a neutral person implementing the program.

IV-13

Statistical Regression

• When participants extremely high or low scores move toward the mean during the post-test.

Solution: Study participants should enclude a random sample of scores that represent the full range of scores possible.

EV-1

Selection Treatment Interaction

• When the results of a study are overgeneralized to other groups for which the study results may not be applicable.

Solution: Researchers should not attempt to generalize results beyond the study poplualtion.  Using random groupings of students instead of intact groups also will assist in the ability to generalize results.

EV-2

Setting Treatment Interaction

• The extent to which the environment conditions of the study can be replicated to other locations.

Solution: Ensure the groups/settings/conditions being compared are similar to increase the external validity.

EV-3

History Treatment Interaction

• This occurs when the results of a study are generalized to past or future situations.

Solution: Many experiments are time sensitive and can not be generalized to other periods.  Researchers need to examine the time periods and affects they may have prior to generalizing results. Example: Weight loss success after January 1st compared to starting the study in September.

1. Randomly select subjects.
2. Randomly assign them to groups.
3. Create a control group
4. Incorporate the use of Analysis of Covariance (ANCOVA) to equalize differences in groups.
5. Focus on ensuring internal validity before establishing external validity.