Internal Validity

A study that does a good job controlling for the possible effects of extraneous variables on the dependent variable.

Internal validity exists if the observed effects of the independent variable on the dependent variale are real and not caused by extraneous factors.

To ensure the researcher has internal validity the researcher must control the internal threats such as; history effect, maturation effect, testing effect, instrumentation effect, selection bias, selection maturation effect, statistical regression, mortality/attrition, Hawthorne effect, placebo effect, diffusion of treatment, location, and implementation.(pg. 164)

External Validity

External Validity is concered with teh ability to generalize the study results to other groups and settings beyond those in the current experiment.

The researcher must focus on external validity because research that can be generalized will typically have more practical value to the profession than research that cannot be generalized. Threats to external validity can come from; selection treatment interaction, setting treatment interaction, and history treatment interaction. (pg. 164-165)

Threats to Internal Validity

Extraneous factors that allow for alternative explanations as to what caused a given effect on the dependent variable.

Researchers must try to control for them to achieve internal validity.  The common threats to internal validity are; history, maturation, testing, instrumentation, selection bias, selection maturation, statistical regression, mortalitiy/attrition, Hawthorne, placebo, diffusion, location, and implementation. (pg. 165)

The History Effect

The history effect is a threat to internal validity.  It threatens validity when events occur between the pretest and the posttest of a research that could affect participants in such a way as to impact the dependent variable.

To address this concern it is best to have a control group.  This way whatever events take place outside of the study would be controlled by people who received the intervention and those that didn't. There would still be something to compare the results to at the end of the study.

(pg. 165)

Control Group

A group of participants exactly like the treatment group in every possible way except they receive no treatment.

Using a control group is a reliable way to deal with threats to validity.

(pg. 166)

Experimental Group

The group receiving the treatment is often referred to as the experimental group.

Researchers use the experimental group along with the control group to make comparisons at the end of the study, to account for validity.

(pg. 166)

Maturation Effect

This threat to internal validity occurs when there are changes seen in subjects because of the time that has elapsed since the study began and which may not be the results of any program effects.

To address this concern researchers the researcher should use a control group. Since both the control and experimental group would be expected to change over time any differences between the two groups at the end of the study would most likely be the result of the study.

(pg. 166)

Testing Effect

Learning that results from taking a pretest causes one ot do better on a posttest.The participant may become privvy to the information on the test and do better on the posttest not because they changed as a result of the intervention, but because they remember information from the pretest.

To address this concern researchers can use a control group that does not receive any pretesting so to reduce this threat to internal validity.

(pg. 166)

Baseline

A measure taken to obtain the status or level of a variable prior to initiating a study.

The purpose of the baseline is to compare a posttest measurement to determine the effectiveness of a program.  This can cause problems with internal validity making the subject "test-wise".

(pg. 166)

Test-Wise

A learning technique or strategies for taking a test that improve one's test scores.

When a subject becomes test-wise in a study it can threaten the internal validity.  It can unvalidiate the findings in a test if the subjects' scores change because they were test wise, not because of the intervention or treatment.  To address this concern the researchers can use a control group that was not pretested.

(pg. 166)

Instrumentation Effects on Validity

The instrumentation used to collect data in a study can cause threates to internal validity when measurements are not accurate or procedures are not standardized. For instance, if a thermometer is broken and gives incorrect numbers the results will not be valid.

To address this concern researchers must be sure to use well designed and tested measurement instruments.

(pg. 166-167)

Selection Bias

Threats to validy that occurs when study participants are selected in a nonrandom manner. This can happen when intact groups are used to conduct studies. Any difference found at the end of the study may be hte restult of intitial differences  between the groups and not the program being studied.

Selection bias can't be fully resolved but there are ways researchers can address the concerns. If an intact group or volunteers must be used, they can be randomly assigned to groups.  Matching participants on selcted characteristics and then randomly assigning them to groups can also reduce selection bias. Lastly, pretesting groups on measures of the dependent variable to make sure there are no pretreatment differences between groups can help address selection bias.

(pg. 167)

Selection Maturation Effect

Another threat to internal validity that actually combines a selection bias with a maturation threat. This occurs when using intact groups that vary in their maturation level.

To address this concern researchers can pretest and/or prescreen groups on maturity level to avoid the threat.

(pg. 166-167)

Statistical Regression

This threat to internal validity occurs when participants are selected on the basis of their extremely high or low scores. If the same instrument was given to them again, the tendency would be for those who scored extremely high on the first test to score lower on the second administration.

To address this concern the researchers can use random samples representing every score rather than just the high and low scores.

(pg. 168)

Mortality Effect

A threat to internal validity from an attrition of study participants.

There is no good way to address this concern.  If a participant drops out or dies that was randomly assigned it changes the validity of the study.

(pg. 168-169)

Hawthorne Effect

When participants' attitudes towrads being involved ina  study affect the way they behave. This was first observed at the Hawthorne Plant of Western Electric Company in Chicago - thus the name.

This threat to internal validity can be addressed could possibly handled by using a control group that receives treatment that doesn't have a direct effect on the variable. This can sometimes be hard to come up with.  Alternatively, the researchers can keep the observed from knowing they are taking part in a study or being observed. This can cause issues with full disclosure so is not always a viable tactic.

(pg. 196)

Placebo Effect

A threat to internal validity that is caused by participants' expectations rather than by any provided treatment.

To control for this researchers can try to make sure  those in both the control and experimental groups receive the same information so both groups have similar expectations.  Another way to handle this is to give both groups as little information as possible to reduce expectations.

(pg. 169-170)

Placebo

Something that looks like the actual treatment, but in reality has no effect; often used in drug studies.

Researchers must look out for the placebo effect in regards to internal validity. Where participants believe they are having the effects because they are receiving the placebo, but in actually are not.

(pg. 170)

Blind Study

Those involved in the study do not know whether they are receiving the treatment or placebo.

Researchers must be cautious of the placebo effect in regards to internal validity.

(pg. 170)

Double Blind Study

If the researcher also does not know who is taking the real drug and who is taking the placebo.

This can help with internal validity becuse any differences seen between the real drug and the placebo can be attributed to the drug and not the expectations of the particpants or the researcher.

(pg. 170)

Diffusion Effect

A threat to internal validity that occurs when the treatment being applied to one group spills over or contaminates another group.

This is a difficult concern to address.  If possible it is best to use to completely unrelated groups so the effects can't spill over.

(pg. 170)

Location Effect

This threat to internal validity occurs when there are differences in locations where interventions take place. 

The best way to address this concern is to make the locations the same for all participants.  If not feasible, the researcher should do everything possible to minimize location differences that could impact the dependant variable.

(pg. 170-171)

Implementation Effect

This threat to internal validity involves the individual or individuals responsible for implementing the experimental treatment and the posibility that they may inadvertently introduce inquality or bias into the study. It can also occur if an individual implementing the intervention inadvertently favors one group over another.

To address this concern researchers should try to make all persons responsible for implementing the program are equally trained and competent, and by following a standardized protocol for implementation. Another strategy is to have all individuals involved in the implementation instruct or present to all groups.  To address the second way it can occur, whenever possible, the researchers shoudl have someone other than the program developer present the program.

(pg. 171-172)

Selection Treatment Interaction

This threat to external validity concerns the ability of a researcher to generalize the results of a study beond the groups involved in the study.

To address this concern the researcher needs to select the most representative group they can collect and only generalize to the sample that is representative of the inteded population on at least some level. The larger the sample, the higher the external validity, but the larger the sample the higher the level of difficulty to carry out the research.

(pg. 172-173)

Study Population

Those in the population who are accessible to the researcher, who had the potential of being selected for the study, and to whom the results can be generalized.

To control for selection treatment interactions' threat to external validity - the researcher should really only generalize to the study population.

(pg. 173)

Population

Those individuals in the universe that are specified by time and place.

Researchers need to make sure they don't generalize to entire populations when the complete a study because they were not all part of the study population so the results don't necessarily include them.

(pg. 173)

Setting Treatment Interaction

This threat to external validity concerns the extent to which the environmental conditions or setting under which an experimental study was conducted can be duplicated in other settings.

To address this concern the researchers must again be cautious of who the population and study population is and only generalize to those in the study population.

(pg. 174-175)

History Treatment Effect

This threat to external validity develops when the researcher tries to generalize findings to past and future situations.  Some experiments may be time-sensitive and may not produce similar results if conducted earlier or later.

Again, to address this concern researchers need to be wary of who the population and study population are and only generalize when appropriate.

(pg. 175)

Analysis of Covariance (ANCOVA)

A statistical technique that can be used to control for initial group differences.

This is important when researchers need internal and external validity. Without internal validity, external validity cannot exist. Thus - the importance of the ANCOVA.

(pg. 175)

Valid

Results are technically sound and accurate. In other words, the results represent real differences and are not due to extraneous variables, statistical manipulations, or chance.

It is extremely important to achieve internal and external validity for researches to have worth.

(pg. 164)

Laboratory Research

Experimental research that takes place in a laboratory setting.

Researches must choose which method of research they will need to achieve the most validity.

(pg. 164)

Field Research

Experimental research that takes place in a more "natural" or "real-life" setting.

Again, researchers must choose which method of research works best for what they are aiming to study.

(pg. 164)