Term
|
Definition
Definition of a concept or variable on the basis of the specific measures used in the research project. The operational definition provides the working definition, however every definition has limits.
a) The definition captures only a portion of the concept
b) Thedefinitionmayincludesomeirrelevantcorrelatedfeatures
c) The use of a single measure means the definition relies heavily upon the specific and unique features of the measure. E.g., intelligence would be defined as a person’s score on the WAIS.
To define a concept in such a way that it can be measured.
Vogt, 220: (a) a description of the way researchers will observe and measure a variable; so called because it specifies the actions (operations) that will be taken to measure the variable. (b) The criteria used to identify a variable or condition.
Operational definitions are essential. They make intersubjectivity (objectivity) possible because they can be replicated; operational definitions are always imperfect, usually being artificial or too narrow. E.g., the operational definition of an overweight person could be whose body mass index (BMI) is over 25. However, some slightly muscled individuals have BMIs over 25 but would not fit most other definitions of overweight. |
|
|
Term
|
Definition
Research strategies that are designed to provide or allow a full and thorough description of a specific sample. Not intended to make between group comparisons or to provide inferential data. Descriptive only.
Vogt, 256: Said of research designs commonly used to study qualitative data. The distinction between qualitative and quantitative design is hard to maintain. Virtually every major research design can be employed to gather either qualitative or quantitative data. E.g., surveys are usually thought of as a quantitative design or method, but that is not necessarily the case. Surveys might ask respondents to answer questions on a Likert scale, which are then summed into a quantitative index. But respondents could just as easily be asked open-ended questions; their answers would become texts that could be studied with grounded theory, which is a qualitative method of analysis. |
|
|
Term
|
Definition
research strategies that are designed to provide inferential statistics from a sample. These statistics can then be generalized to a target population which the original sample was supposed to represent.
Vogt, 256: Said of variables or research that can be handled numerically. Usually contrasted (too sharply) with qualitative variables and research. Many research designs lend themselves well to collecting both quantitative and qualitative data, and many variables can be handled either qualitatively or quantitatively. E.g., naturalistic observations can give rise to either or both kinds of data. Interactions can be counted and timed with a stopwatch or they can be interpreted more holistically. |
|
|
Term
|
Definition
Research which examines a treatment or phenomena under conditions which approximate the “real world” or clinical settings. E.g., bio-dome experiments- artificial settings to look at a natural behavior; there is some control over the environment.
Vogt, 8: (Also spelled: analog) Said of data or computers that use a system of representation that is physically analogous to the thing being represented. E.g., a thermometer can use the height of a column of mercury to indicate heat; the higher the column, the higher the temperature. Or, analog watches use the physical movement of hour and minute hands to represent the passing of time; the more hands have moved, the more time has passed. |
|
|
Term
|
Definition
research strategies that provide the clearest case for drawing causal inferences. These designs have the greatest control over independent variables and sources of error. The most common characteristics include: Random assignment of subjects to conditions Manipulation of the independent variable Use of statistical analyses (ANOVA, MANOVA, ANCOVA) Use of a control group These designs include between-subjects as well as within-subjects (repeated measures) independent variables.
Vogt, 328: An experiment. “True” is contrasted with the methods of quasi-experiments and natural experiments. The key distinction is that, unlike in other research designs, in a true experiment subjects are randomly assigned to treatment groups and the researchers manipulate the independent variables. |
|
|
Term
|
Definition
each person serves in only one group and the groups are compared.
Between-Subjects Designs or (ANOVA) (Vogt, 25) A research procedure that compares different subjects. Each score in the study comes from a different subjects. Usually contrasted to a within- subjects design, which compares the same subjects at different times or under different treatments. |
|
|
Term
|
Definition
each person serves in every condition. For instance, “time” is the most common within-subjects variable whereby you compare each person’s pre-test to their own post-test.
Within-Subjects Design (Vogt, 343) A before-and-after study or a study of the same subjects given different treatments. A research design that pretests and posttests within the same group of subjects, that is one which uses no control group. |
|
|
Term
|
Definition
| Repeated-Measures Design (Vogt, 274) . A design in which subjects are measured two or more times on the dependent variable. Rather than using different subjects for each level of treatment, the subjects are given more than one treatment and are measured after each. This means that each subject is it's own control. |
|
|
Term
| Quasi-Experimental Design |
|
Definition
Designs that approximate the control offered by experimental designs, but where true random assignment to all groups is not possible
However, true random assignment to conditions is not possible. Characteristics include:
• A Static variable that does not allow random assignment of subjects to groups/conditions. The
condition is such that you cannot randomize.
• Manipulation of at least one independent variable
• Statistical analyses (ANOVA, Regression, MANOVA, ANCOVA)
Vogt, 257: A type of research design for conducting studies in field or real-life situations where the researcher may be able to manipulate some independent variables but cannot randomly assign subjects to control and experimental groups. E.g., you cannot cut off some individuals’ unemployment benefits to see how well they could get along without them or to see whether an alternative job training program would be more effective. But you could try to find volunteers for the new job training program. You could compare the results for the volunteer group (experimental group) with those of people in the regular program (control group). The study is quasi-experimental because you were unable to assign subjects randomly to treatment and control groups. |
|
|
Term
| Correlational Designs, Static Group/Case Control |
|
Definition
Designs in which the variable of interest is studied by selecting participants who vary in their display of the variable or characteristic of interest. This design is more observational/descriptive because it does not allow for causal inferences.
• Independent variables are not manipulated!
• Participants are not randomly assigned!
• Statistical analysis used include Chi-Square, Regression, and Correlations
• Observational or Descriptive- Does not allow for inferences about causal relationships.
Vogt, 64: A design in which the variables are not manipulated. Rather, the researcher uses measures of association to study their relations. The term is usually used in contrast with experimental research. |
|
|
Term
|
Definition
Vogt, 107: Any uncontrolled or unexplained variability, such as within-group differences in an ANOVA. Also called “random error,” “random variance,” and “residual.” The error variance is the variance of the error term.
Severino: This is all the variations within each condition of your independent variable (i.e. experimental group and control group). It basically represents all the individual differences among participants (differences between individuals that will keep you from seeing the true effect of the experiment) in each group and is assumed to be randomly distributed. This “noise” or variance can increase the probability of a Type II error (not finding significant differences when they exist). |
|
|
Term
|
Definition
| all the ways something can differ. For instance, if you looked at the income in La Jolla the variance would probably be rather small whereas if you included all areas of San Diego County the variance would be much broader! |
|
|
Term
|
Definition
Also known as secondary variance. This refers to differences that are NOT randomly distributed across groups. A covariate is a variable that is a potential confound in the study and you have found that it does, in fact, affect your dependent variable! This is a huge threat to your internal validity! Covariance is also described as systematic differences between groups that are not accounted for by the treatment effect!
A measure of the joint or (co-) variance of two or more variables. A covariance is an unstandardized correlation coefficient r and it is more often reported as an r. However, the covariance is very important in calculating many multivariate statistics. [Example in PDF]. |
|
|
Term
|
Definition
| group that you are interested in generalizing your results to. Parameters must be specific. |
|
|
Term
|
Definition
| the selected subset of your population (sample mean = population mean). For instance, your population may be all college students but your sample would only include those at AIU, UCSD, and SDSU due to sampling difficulties! |
|
|
Term
|
Definition
The extent to which the intervention or manipulation of the IV can be considered to account for the results, changes or group difference, rather than extraneous influences. The extent to which an experiment rules out alternative explanations.
The extent to which the results of a study (usually an experiment) can be attributed to the treatments rather than to flaws in the research design. In other words, internal validity is the degree to which one can draw valid conclusions about the causal effects of one variable on another. It depends on the extent to which extraneous variables have been controlled by the researcher. |
|
|
Term
|
Definition
| The extent to which the results can be generalized beyond the conditions of the research to other populations, settings or conditions. Also known as generalizability. |
|
|
Term
|
Definition
The joint effect of two or more independent variables on a dependent variable. Interaction effects occur when independent variables not only have separate effects, but also have combined effects that are different from the simple sum of their separate effects. In other terms, interaction effects occur when the relation between two variables differs depending on the value of another variable. The presence of statistically significant interaction effects makes it difficult to interpret main effects. Also called “conditioning effect,” “contingency effect,” “joint effect,” and
“moderating effect.”
When two variables interact, this is called a first-order interaction; when three interact, it is a second- order interaction, and so on. Interaction effects may be ordinal or disordinal. [EXAMPLE in pdf] |
|
|
Term
|
Definition
| The simple effect of an independent variable on a dependent variable; the effect of an independent variable uninfluenced by (without controlling for the effects of) other variables. Used in contrast with the interaction effect of two or more independent variables on a dependent variable. It is difficult to interpret main effects in the presence of interaction effects. |
|
|
Term
|
Definition
| A tentative answer to a research question; a statement of (or conjecture about) the relationships among the variables that a researcher intends to study. Hypotheses are sometimes testable statements of relations. In such cases, they are usually thought of as predictors, which if confirmed, will support a theory.[example in PDF] |
|
|
Term
|
Definition
In hypothesis testing any hypothesis that does not conform to the one being tested, usually the opposite of the null hypothesis. Also called the research hypothesis. Rejecting the null hypothesis shows that the alternative (or research) hypothesis may be true.
E.g., researcher conducting a study of the relation between teenage drug use and teenage suicide would probably use a null hypothesis something like: “There is no difference between the suicide rates of teenagers who use drugs and those who do not.” The alternative hypothesis might be: “Drug use by teenagers increases their likelihood of committing suicide.” Finding evidence that allowed the rejection of the null hypothesis (that there was no difference) would increase the researchers’ confidence in the probability that the alternative hypothesis was true. |
|
|
Term
|
Definition
An hypothesis that a researcher usually hopes to reject, thereby substantiating its opposite. Often the hypothesis that two or more variables are not related or that two or more statistics (e.g., means for two different groups) are not the same. The “null” does not necessarily refer to zero or no difference (although it usually does); rather it refers to the hypothesis to be nullified or rejected. In accumulating evidence that the null hypothesis is false, the researcher indirectly demonstrates that the variables are related or that the statistics are different. The null hypothesis is the core idea in hypothesis testing.
The null hypothesis is something like the presumption of innocence in a trial; to find someone guilty, the jury has to reject the presumption of innocence. To continue with the analogy, they have to reject it beyond a reasonable doubt. The “reasonable doubt” in hypothesis tests is the alpha level. |
|
|
Term
|
Definition
the problem to be investigated in a study stated in the form of a question. It is crucial for focusing the investigation at all stages, from the gathering through the analysis of evidence. A research question is usually more exploratory than a research hypothesis or a null hypothesis
E.g., a research question might be: What is the relation between A and B? A parallel research hypothesis could be: Increases in A lower the incidence of B. The associated null hypothesis might be: There is no difference in the mean levels of B among subjects who have different levels of A. |
|
|
Term
|
Definition
The classical approach to assessing the statistical significance of findings. Basically it involves comparing empirically observed sample findings with theoretically expected findings- expected if the null hypothesis is true. This comparison allows one to compute the probability that the observed outcomes could have been due to chance or random error.
E.g., suppose you wanted to study the effects on performance of working in groups as compared to working alone. You get 80 students to volunteer for your study. You assign them randomly into two categories: those who work in teams of four students and those who would work individually. You provide subjects with a large number of math problems to solve and record the number of answers they got right in 20 minutes. Your alternative or research hypothesis might be that people who work in teams are more efficient than those who work individually. To examine the research hypothesis, you would try to find evidence that would allow you to reject your null hypothesis- which would probably be something like: There is no difference between the average score of students who work individually and those who work in teams. |
|
|
Term
|
Definition
| An error made by wrongly rejecting a true null hypothesis. This might involve incorrectly concluding that two variables are related when they are not, or wrongly deciding that a sample statistic exceeds the value that would be expected by chance. Also called alpha error or false positive. |
|
|
Term
|
Definition
| an error made by wrongly retaining (or accepting or failing to reject) a false null hypothesis. Also called beta error or false negative. |
|
|
Term
| Type I and Type II Errors |
|
Definition
| inversely related; the smaller the risk of one, the greater the risk of the other. The probability of making a Type I error can be precisely computed in advance for a particular investigation, but the exact probability of Type II error is generally unknown. |
|
|
Term
|
Definition
Loosely, anything studied by a researcher. Any finding that can change, that can vary, that can be expressed as more than one value or in various values or categories. The opposite of a variable is a constant.
Examples of variables include anything that can be measured or assigned a number, such as unemployment rate, religious affiliation, experimental treatment, GPA, etc. Much of social science is aimed at discovering and explaining how differences in some variables are related to differences in others. |
|
|
Term
|
Definition
in a study. Also a variable that can be used to . A variable manipulated by an experimenter who predicts that the manipulation will have an effect on another variable (the dependent variable).
Some authors use the term “independent variable” for experimental research only. For these authors the key criterion is whether the researcher can manipulate the variable; for nonexperimental research these authors use the term “predictor variable” or “explanatory variable.” However, most writers us “independent variable” when they many any causal variable, whether in experimental or nonexperimental research.
Examples of IV: (Severino)
Environmental/Situational: Varying what is done to, with, or by the subject. E.g., a task is provided to some but not to others. (Is CBT more effective than Mindfulness Meditation in treating depression? Therapy is the IV here and it has 2 levels = meditation and CBT)
The presumed cause

predict or explain the values of another variable
Classifies the common bond between groups we are comparing (environmental/situational,
static, instructional), the construct, experimental manipulation, intervention, or factor whose impact
will be evaluated in the study

Instructional: variations in what participants are told or led to believe through verbal or written statements, usually aimed at altering the perception of the situation (Do therapists interpret psychological test results differently when they are told that the test responses were produced by disturbed patients versus people who are functioning well?)
Organismic (Static): an independent variable that cannot be manipulated so subjects can not be assigned randomly to these conditions. For instance, gender could be an IV, however, you cannot randomly assign a person to be male or female!! Other static variables that may be of interest as IVs could be year level in grad school or generation/age. (Do men score higher on a measure of physical aggression than do women?) |
|
|
Term
|
Definition
| An extraneous variable that you do not wish to examine in your study; hence you control for it. Also called covariate. (Vogel) extraneous variable held constant because it could offer an alternative explanation for the results (more in threats to internal validity). Examples include age, gender, time of day, environment. |
|
|
Term
|
Definition
The presumed effect in a study; so called because it “depends” on another variable. The variable whose values are predicted by the independent variable, whether or not they are caused by it. Also called “outcome,” “criterion,” and “response” variable.
E.g., in a study to see if there were a relationship between students’ drinking of alcoholic beverages and their GPA, the drinking behavior would probably be the presumed cause (independent variable); the GPA would be the effect (dependent variable). But, it could be the other way around, if, for instance, one wanted to study whether students’ grades drive them to drink. |
|
|
Term
|
Definition
| A variable that distinguishes among subjects by sorting them into a limited number of categories, indicating type or kind, as religion can be categorized: Buddhist, Christian, Jewish, Muslim, Other, None. Breaking a continuous variable, such as age, to make it categorical is a common practice, but since this involves discarding information it is usually not a good idea. Also called, “discrete” or “nominal” variable. |
|
|
Term
|
Definition
A variable that can be expressed by a large (often infinite) number of measures. Loosely, a variable that can be measured on an interval or ratio scale. While all continuous variables are interval or ratio, all interval or ratio scales are not continuous, in the strict sense of the term.
Deciding whether to treat data as continuous can have important consequences for choosing statistical techniques. Ordinal data are often treated as continuous when there are many ranks in the data, but as categorical when there are few.
E.g., height and GPA are continuous variables. People’s heights could be 69.38 inches, 69.39 inches, and so on; GPAs could be 3.17, 3.18, and so on. In fact, since values always have to be rounded, theoretically continuous variables are measures as discrete variables. There is an infinite number of values between 69.38 and 69.39 inches, but the limits of our ability to measure or the limits of our interest in precision lead us to round off continuous values.
GPA is a good example of the difficulty of making these distinctions. It is routinely treated as a continuous variable, but it is constructed out of a rank order scale (A, B, C, etc.). Numbers are assigned to those ranks, which are then treated as thought they were an interval scale. |
|
|
Term
|
Definition
| A variable that varies in ways the researcher does not control; a variable whose values are randomly determined. “Random” refers to the way the events, values, or subjects are chosen or occur, not to the variable itself. Men and women are not random, but sex could be a random variable in a research study; the sex of subjects included in the study could be left to chance and not controlled by the researcher. Also called stochastic variable. |
|
|
Term
| What is the simple difference between an IV, DV, and Covariate or Control Variable? |
|
Definition
IV = MANIPULATED!!!
DV = OBSERVED / MEASURED
Control Variable/Covariate = Held Constant |
|
|
Term
|
Definition
An extension of ANOVA that provides a way of statistically controlling the (linear) effects of variables one does not want to examine in a study. These extraneous variables are called covariates, or control variables. ANCOVA allows you to remove covariates from the list of possible explanations of variance in the dependent variable. ANCOVA does this by using statistical techniques (such as regression) to partial out of the effects of covariates rather than direct experimental methods to control extraneous variables.
ANCOVA is used in experimental studies when researchers want to remove the effects of some antecedent variable. E.g., pretest scores are used as covariates in pre-/posttest experimental designs. ANCOVA is also used in nonexperimental research, such as surveys of nonrandom samples, or in quasi-experiments when subjects cannot be assigned randomly to control and experimental groups. Although fairly widespread, the use of ANCOVA for nonexperimental research is controversial. All ANCOVA problems can be handled with multiple regression analysis using dummy coding for the nominal variables, and, with the advent of powerful computers, this is a more efficient approach. Because of this, ANCOVA is used less frequently than in the past. |
|
|
Term
|
Definition
(Vogt, 144) To “subtract” the effects of a variable from a complex relationship so as to study what the relationship would be if the variable were in fact a constant. Holding a variable constant essentially means assigning it an average value.
E.g., in a study of managerial behaviors and their effects on workers’ productivity, a researcher might want to hold the education of the managers constant. This would especially be the case if she had reason to believe that different kinds or amounts of education might lead managers to behave differently. |
|
|
Term
|
Definition
A research design in which subjects are matched on characteristics that might affect their reaction to a treatment. After the pairs are determined, one member of each pair is assigned at random to the group receiving treatment (experimental group); the other group (control group) does not receive treatment. Without random assignment, matching is not considered good research practice. Also called “subject matching.”
E.g., if professors wanted to test the effectiveness of two different textbooks for an undergraduate statistics course, they might match the students on quantitative aptitude scores before assigning them to classes using one or another of the texts. An alternative, if the professors had no control over class assignment, would be to treat the quantitative aptitude scores as a covariate and control for it using an ANCOVA design. |
|
|
Term
|
Definition
| An experimental design in which subjects are grouped into categories or “blocks.” These blocks may then be treated as the experiment’s unit of analysis. The goal of categorizing subjects into blocks is to control for a covariate. See matched pairs and randomized block design. |
|
|
Term
|
Definition
A research design in which subjects are matched on a variable the researcher wishes to control. The subjects are put into groups (blocks) of the same size as the number of treatments. The members of each block are assigned randomly to different treatment groups. Compare Latin square, repeated-measures ANOVA.
E.g., say we are doing a study of the effectiveness of four methods of teaching statistics. We use 80 subjects and plan to divide them into 4 treatment groups of 20 students each. Using a randomized- blocks design, we give the subjects a test of their knowledge of statistics. The four who score highest on the test are the first block; the next highest four are the second block, and so on to the 20th block. The four members of each block are randomly assigned, one to each of the four treatment groups. We use the blocks to equalize the variance within each treatment group by making sure that each group has subjects with a similar prior knowledge of statistics. |
|
|
Term
| Counterbalancing Technique |
|
Definition
In a within-subjects factorial experiment, presenting conditions (treatments) in all possible orders to avoid order effects. See Latin square.
E.g., an experimenter might wish to study the effects of three kinds of lighting (A, B, and C) on performance on a visual skill. Subjects could first be placed in Condition A and be given a test of the skill; then they could be put in Condition B and get a second test, and so on. By Condition C and the third test, subjects’ scores might go up simply because they had the practice of the first two tests. Or their scores might go down because they become fatigued.
The effects of practice and fatigue could be counterbalanced by rotating the lighting conditions so that subjects would experience them in all possible orders. Since there are six possible orders (ABC, ACB, BAC, BCA, CAB, and CBA), subjects could be dividing into six groups, one for each possible order. |
|
|
Term
|
Definition
A means of reducing bias in an experiment by insuring that both those who administer a treatment and those who receive it do not know (are “blind” to) which subjects are in the experimental and control groups, that is, who is and who is not receiving the treatment.
E.g., in a study of the effectiveness of various headache remedies, 80 headache sufferers could be randomly assigned to four groups. Group A would receive aspirin, Group B ibuprofen, Group C acetaminophen, and Group D a placebo. The pills might be color-coded, but otherwise look the same so that the experimenter handing them out would not know which subjects were getting which; and, of course, the subjects would not know. When subjects experienced pain, they would be given pills depending upon their group and then asked about the relief they got from their pills. Their responses would be data used to evaluate the effectiveness of the various remedies. If the experiment used true double-blind procedures, the researchers analyzing the data would not know, until after they had reached their conclusions, which group received which remedy; they would only know, say, that on average blue pills worked better than red ones. |
|
|
Term
|
Definition
In experimental research, a group that, for the sake of comparison, does not receive the treatment the experimenter is interested in studying. Compare experimental group.
E.g., psychologists studying the effects of TV violence on attitudes might give subjects a questionnaire to measure their attitudes, divide the group into tow, show a videotape on a violent program to one half (the experimental group), and show a nonviolent program to the other half (the control group). A second attitude questionnaire would then be given to the two groups to see whether the programs affected their scores.
a) (Severino) Control groups usually employed to address threats to internal validity (i.e., history, maturation, selection, testing, etc.).
b) You want the control group to share these influences with the experimental group, but does not receive the treatment. |
|
|
Term
| No-Treatment Control Group |
|
Definition
A group that is assessed, but does not receive treatment or intervention.
• Use of no-tx control directly controls the effects of history, maturation
• Asses the base rate of improvement for clients who do not receive the treatment under
investigation
• Ethical considerations include problems associated with withholding tx. |
|
|
Term
|
Definition
Like a no-tx control, but treatment is only withheld temporarily
The period of time that tx is withheld usually corresponds to the pre to post test assessment interval. As soon as the second assessment battery is administered, the wait- list subjects receive their tx.
Because subjects in wait-list controls receive treatment after the post-test period, long- term follow up is not possible since the control is no longer available for comparisons. |
|
|
Term
| Attention Placebo Control Group |
|
Definition
Designed to control for common factors that are associated with participation in treatment (e.g., Hawthorne effects)
• The goal is to provide a pseudo intervention that involves participants in some sort of experience, thus controlling for factors common to coming to treatment (i.e., attending sessions, meeting with a professional, etc.) account for the change.
• (Vogel) Both the experimental and control group have equivalent contact with research personnel (pseudo intervention) to control for placebo effects. |
|
|
Term
|
Definition
Used in studies in which differences in procedures may arise as a function of implementing a particular intervention.
• Purpose is to ensure that groups are equal with respect to potentially important but conceptually and procedurally irrelevant factors.
• Pairs are formed arbitrarily (unless matching was used to assign to groups). A subject in the experimental group receives a certain number of sessions on the basis of his/her progress. The yoked-control participant receives the same number of sessions.
• (Vogel) Used when the procedure changes for each subject based on their performance, pairs are arbitrarily formed so that the subject in the experimental group and the yoked- control subject receive same number of sessions/trials/etc. |
|
|
Term
|
Definition
| Groups that are added to an experiment that utilize subjects who were not part of the original subject pool and not randomly assigned to treatment. |
|
|
Term
| Sampling Distribution of a Statistic |
|
Definition
A theoretical frequency distribution of the scores for or values of a statistic, such as a mean. Any statistic that can be computed for a sample has a sampling distribution is the distribution of statistics that would be produced in repeated random sampling (with replacement) from the same population. It is composed of all possible values of a statistic and their probabilities of occurring for a sample of a particular size.
A sampling distribution is constructed by assuming that an infinite number of samples of a given size have been drawn from a particular population and that their distributions have been recorded. Then the statistic, such as the mean, is computed for the scores of each of these hypothetical samples; then this infinite number of statistics is arranged in a distribution to arrive at the sampling distribution. The sampling distribution is compared with the actual sample statistic to determine if that statistic is or is not likely to be the size it is due to chance.
It is hard to overestimate the importance of sampling distributions of statistics. The entire process of inferential statistics (by which we move from known information about samples to inferences about populations) depends on sampling distributions. We use sampling distributions to calculate the probability that sample statistics could have occurred by chance, and thus to decide whether something that is true of a sample statistic is also likely to be true of a population parameter.
(Vogel) The sampling distribution is a distribution used as a model of what would happen if:
• The null hypothesis were true (there really were no effects), and
• The experiment was repeated an infinite number of times.
May 2007
METHODOLOGICAL CONSIDERATIONS 18
Sampling distributions are created using Monte Carlo experiments whereby a large number of equal sized random samples are drawn from a population you wish to represent. For each sample, the statistic is computed and the stats are arranged in a frequency distribution so you can see the normal curve for that population. Doing these samples over and over again allow you to finally get that population sampling distribution.
(Vogt, 196) Any generating of random values (most often with a computer) in order to study statistical models. Monte Carlo methods involve producing several sets of artificial data and using these to study the properties of estimators. E.g., |
|
|
Term
|
Definition
whereby a large number of equal sized random samples are drawn from a population you wish to represent. For each sample, the statistic is computed and the stats are arranged in a frequency distribution so you can see the normal curve for that population. Doing these samples over and over again allow you to finally get that population sampling distribution.
(Vogt, 196) Any generating of random values (most often with a computer) in order to study statistical models. Monte Carlo methods involve producing several sets of artificial data and using these to study the properties of estimators. E.g., statisticians who develop a new theory want to test it on data. They could collect real data, but it is much more cost-efficient, initially at least, to test the theory on sets of data (often hundreds of sets of data) generated by a Monte Carlo computer program. |
|
|
Term
|
Definition
Whenever a sample is drawn, by definition, only that part of the population that is included in the sample is measured, and is used to represent the entire population. Hence, there must always be some error in the data, resulting from those members of the population who were not measured. Error will, therefore, be reduced as the sample size is increased, so that, if a census is performed (a 100 percent sample is a census), by definition there will be no sampling error.
If a population from which a sample is drawn is large, then population values will not be affected very much by one or two members of the population who have extreme values of a particular measure. For example, the population of the U.S. is about 100 million households; most households have between 1 and 6 members; if 100,000 households have 10 members, and the remaining 99.9 million average 2.300 persons per household, the average for all households will change from 2.300 to 2.308. As a result of this, samples that are quite small in numeric size and very small as a percentage of the population will often have very small sampling errors.
statisticians who develop a new theory want to test it on data. They could
collect real data, but it is much more cost-efficient, initially at least, to test the theory on sets of data
(often hundreds of sets of data) generated by a Monte Carlo computer program.

Sampling error of a mean value estimated from a sample is equal to the estimated standard deviation of the variable divided by the square root of the sample size. It is therefore not dependent on the population
size, but only on the variability of the variable of concern and sample size. For example:
• If one drew a sample of four observations from a large population, the sampling error would be equal to the standard deviation divided by 2 (the square root of four).
• To halve the sampling error of that variable, one would need to increase the sample to 16; it could be halved again by increasing the sample size to 64; and halved again by increasing the sample to 256.
• If a sample of 1,024 were selected, the sampling error would be 1/32 of the standard deviation; because standard deviations on many variables are fairly small values, this represents a very small error.
It also follows that increasingly large increases in sample size are necessary to continue to decrease the sampling error - to halve the error again to 1/64th of the standard deviation would require an increase of the sample size to 4,096, while halving again would require an increase to 16,384. |
|
|
Term
|
Definition
| Another term for “random sampling.” “Selection” is more often used in experimental research; “sampling” is the more common term in survey research, but the underlying principle is the same. See random assignment, probability sample. |
|
|
Term
|
Definition
Selecting a group of subjects (a sample) fro study from a larger group (population) so that each individual (or other unit of analysis) is chosen entirely by chance. When used without qualification (such as stratified random sampling), random sampling means
“simple random sampling.” Also sometimes called “equal probability sample,” since every member of the population has an equal probability of being included in the sample. A random sample is not the same thing as a haphazard or accidental sample. Using random sampling reduces the likelihood of bias. Compare probability sample, cluster sample, quota sample, stratified random sample. |
|
|
Term
|
Definition
| Selecting elements (subjects or other units of analysis) from a population in such a way that they are representative of the population. This is done to increase the likelihood of being able to generalize accurately about the population. Sampling is often a more accurate and efficient way to learn about a large population than a census of the whole population. |
|
|
Term
|
Definition
| A sample of subjects selected for a study not because they are representative but because it is convenient to use them - as when college professors study their own students. Compare accidental sample, bias. Oddly, using this term sometimes tends to legitimize bad practice. Students of mind occasionally say, “I used a convenience sample to gather the data for my project,” as though this hard-to- justify method were a reasonable option among the many types of samples such as random systematic, stratified, and so on. |
|
|
Term
|
Definition
A technique for finding research subjects. One subject gives the researcher the name of another subject, who in turn provides the name of a third, and so on. This is an especially useful technique when the researcher wants to contact people with unusual experiences or characteristics who are likely to know one another- members of a small religious
group, for example. Also called networking sample. “word of mouth” |
|
|
Term
| Stratified Random Sampling |
|
Definition
A random or probability sample drawn from particular categories (or "strata") of the population being studied. The method works best when the individuals within the strata are highly similar to one another and different from individuals in other strata. Indeed, if the strata were not different from one another, there would be no point in stratifying. The strata have a function similar to blocks in randomized blocks designs. Stratified random sampling can be proportionate, so that the size of the strata corresponds to the size of the groups in the population. It can also be disproportionate, as in the following example.
Suppose you wanted to compare the attitudes of Protestants, Catholics, Muslims, and Jews in a population in which those four groups were not present in equal numbers. If you drew a simple random sample, you might not get enough cases from one of the groups to make meaningful comparisons. To avoid this problem you could select random samples of equal size within each of the four religious groups (strata). |
|
|
Term
| Proportional Stratified Random Sampling |
|
Definition
| A stratified random sample in which the proportion of the subjects in each category (stratum) is the same as in the population. Compare quota sample. |
|
|
Term
|
Definition
A method for drawing a sample from a population in two or more stages. It is typically used when researchers cannot get a complete list of the members of a population they wish to study, but can get a complete list of groups or clusters in the population. It is also used when a random sample would produce a list of subjects so widely scattered that surveying them would be prohibitively expensive. Generally the researcher wishes to use clusters containing subjects as diverse as possible. By contrast, in stratified sampling the goal is often to find strata containing subjects as similar to one another as possible.
The disadvantage of cluster sampling is that each stage of the process increases sampling error. The margin of error is therefore larger in cluster sampling than in simple or stratified random sampling; but, since cluster sampling is usually much easier (cheaper), this error can be compensated for by increasing the sample size. See central limit theorem.
E.g., suppose you wanted to survey undergraduates on social and political issues. There is no complete list of all college students. But there are complete lists of all 3,000+ colleges in the country. You could begin by getting such a list of colleges (which are “clusters” of students).
You could then select a probability sample of, say, 100 colleges. Once the clusters (colleges) were identified, the researchers could go to each school and get a list of its students; students to be surveyed would be selected (perhaps by simple random sampling) from each of these lists. |
|
|
Term
|
Definition
| A sample in which each case that could be chosen has a known probability of being included in the sample. Often a random sample, which is an equal probability sample. At some point, random selection is part of the process of every probability sample. |
|
|
Term
|
Definition
A stratified non-random sample, that is, a sample selected by dividing a population into categories and selecting a certain number (a quota) of respondents from each category. Individual cases within each category are not selected randomly; they are usually chosen on the basis of convenience. Compare accidental sampling, purposive sample, random sampling, stratified random sampling, proportional stratified random sampling.
E.g., interviewers might be given the following assignment: “Go out and interview 20 men and 20 women, with half of each 50+ years old.” Despite its superficial resemblance to stratified random sampling, quota sampling is not a reliable method to use for making inferences about a population. |
|
|
Term
|
Definition
| A sample gathered haphazardly, e.g., by interviewing the first 100 people you ran into on the street who were willing to talk to you. An accidental sample is not a random sample. The main disadvantage of an accidental sample is that the researcher has no way of knowing what the population might be. See convenience sample, probability sample. |
|
|
Term
|
Definition
A sample composed of subjects selected deliberately (on purpose) by researchers, usually because they think certain characteristics are typical or representative of the population Compare quota sample:
This is generally an unwise procedure; it assumes that the researcher knows in advance what the relevant characteristics are; it runs the risk (because it is not random) of introducing unknown bias. Inferences about a population cannot legitimately be made using a purposive sample. On the other hand, purposive sampling is often the only way to try to increase representativeness in field research, and it can be an improvement over simple convenience sampling. A frequent compromise between random sampling and purposive sampling is stratified random sampling. |
|
|
Term
|
Definition
| Putting subjects into experimental and control groups in such a way that each individual in each group is assigned entirely by chance. Otherwise put, each subject has an equal probability of being placed in each group. Using random assignment reduces the likelihood of bias. Also called random allocation. Goal is group equivalence, but random assignment does not necessarily produce equivalent groups |
|
|
Term
|
Definition
This is used when a subject variable is known to be related to scores on the DV. It is essential to distribute these characteristics across groups so that they do not differ prior to treatment! First, group subjects together based on similarity on the variable in question. Then, randomly assign one person from each pair to the experimental group.
• (Severino) Match subjects with identical pretreatment scores. When 2 are matched perfectly, randomly assign each to either the experimental or control group through a coin toss.
• Rank order subjects from high to low if there are three groups in the experiment, the first three with the highest score form the first block and are assigned randomly to each condition- etc.
• If you want to make sure that groups are equivalent on a categorical variable, have separate lists for each (i.e., male/female) and assign randomly. |
|
|
Term
|
Definition
| A statistical proposition to the effect that the larger a sample size, the more closely a sampling distribution of a statistic will approach a normal distribution. This is true even if the population from which the sample is drawn is not normally distributed. A sample size of 30 or more will usually result in a sampling distribution for the mean that is very close to a normal distributionThe central limit theorem explains why sampling error is smaller with a large sample than with a small sample and why we can use the normal distribution to study a wide variety of statistical problems. |
|
|
Term
| Criteria and Criterion Measure |
|
Definition
| This is another term for dependent variable, criterion are used in correlational research when it is not possible to establish a causal relationship, it is like the outcome of a study |
|
|
Term
|
Definition
|
|
