• RS strategies designed to provide or allow a full and thorough description of a specific sample

• Not intended to make b/n grp. comparisons or to provide inferential data

• Descriptive ONLY

• RS strategies used to provide inferential statistics from a sample to be generalized to a target population which the original sample was supposed to represent

• Variables/RS that can be handled numerically

Operational Definition

"Working Definition"

• Defines a concept in such a way that it can be measured
• Every definition has limits
• Captures only a portion of the concept
• May include irrelevant Information
•  Make intersubjectivity (objectivity) possible BC they can be replicated
• ALWAYS imperfect, usually being artificial or too narrow (eg. Defn of an overweight person as one with BMI >25)

• RS which examines a Tx or phenomena under conditions which approximate the "real world" or clinical settings

-Eg. Bio-dome experiments - artificial settings to look at a natural bx (there is some control over the environment)

• Said of data or computers that use a system of representation that is physically analogous to the thing being represented (Eg. a thermometer can use the height of a column of mercury to indicate heat - the higher the column, the higher the temperature; Analog watches use the physical mov't of hour and minute hands to represent the passing of time - the more the hands have moved, the more time has passed)

1. Random assignment  
2.  Manipulation of the IV
3. Use of statistical analyses (ANOVA, MANOVA, ANCOVA)
4. Control group

• Have the greatest control over independent variables and sources of error
• Provide the clearest case for drawing causal inferences

• Include b/n subjects and within subjects (repeated measures) independent variables

1. b/n subjects

2. b/n subjects or ANOVA

3. within-subjects

4. repeated measures design

• Each person serves in only one group and the groups are compared

• Each score in the study comes from a different subject compares different subjects

• Each person serves in every condition Before and after study or a study of the same subjects given different Txs

• NO Control Group: Compare Pre/posttest within the same group of subjects

• Time = most common within-subjects variable 

   

• Subjects are measured 2+ times on the DV

• Subjects are given >1 Tx and are measured after each level of Tx -

• Each subject serves as its own control

1. Subjects randomly assigned to Tx groups

2. Manipulate the IV

1. NO random assignment
2. Manipulation of at least one IV
3. Statistical analyses (ANOVA, ANOVA, MANOVA)

• Approximate the control offered by experimental designs

• Used in real life or field situations where RSer may be able to manipulate some IVs but cannot truly randomly assign subjects to control and experimental groups (e.g. study with volunteers)

aka "Static Group/Case Control"

1. IV NOT manipulated (Use measures of association to study relations)
2. NO random assignment
3. Statistical analysis used include Chi-Square, Regression, Correlations
4. Observational or descriptive: Does NOT allow for inferences about causal relationships

• Variable of interest is studied by selecting participants who vary in their display of the variable or characteristic of interest

aka Random error, random variance, and residual

• Variance of the error term
• Any uncontrolled or unexplained variability, such as within-group differences in an ANOVA

• All the variations within each condition of your IV (eg. experimental group & control group) -
• Assumed to be randomly distributed
• The "noise" or variance, can increase probability of Type II error (not finding significant differences when they exist)

All the ways something can differ

-eg. Variance within La Jolla in comparison to variance in San Diego

aka "secondary variance"

• Systematic differences b/n groups that are not accounted for by the treatment effect

• Threat to Internal Validity

• Measure of the joint or covariance of 2+ variables

• Differences that are NOT randomly distributed across groups

• Variable that is a potential confound in the study and you have found that it does affect your DV

• Group that you are interested in generalizing your results to
• Parameters must be specific

The selected subset of your population

sample mean = population mean

The extent to which the intervention or manipulation of the IV can be considered to account for the results, changes or group difference, rather than extraneous influences

1. Random assignment
2. Manipulation of the IV
3. Control group
4. Use of statistical analyses (ANOVA, MANOVA, ANCOVA)

• Provide clearest case for drawing causal inferences
• Have greatest control over IVs and sources of error

-True Experimental Design

-Each person serves in only one group and the groups are compared

• True Experimental Design
• Each score in the study comes from a different subject

• Usually contrasted to a within-subjects design, which compares the same subjects at different times or under different treatments

• Each person serves in every condition

• No Control Group: Pre/Posttest from within the same group

• A before-and-after study or a study of the same subjects given different treatments

• Time is the most common within subjects variable

• Subjects are given more than one treatment and measured after each

• Each subject is its own control

• Subjects are measured on the DV

1. Pre-Post Control

2. Post Only Control

3. Solomon 4 Group

4. Time Series

5. Factorial

Experimental Design:

A. Descriptive of Causal?

B. Random Assignment?

C. Static or Manipulated IVs?

D. Control Group?

E. Statistics Used

EXPERIMENTAL DESIGN:

A. Causal

B. Randomly Assigned

C. Both Static & Manipulated IVs

D. Control Group

E. a. t-test

b. ACOVA

c. ANCOVA

d. MANOVA

Quasi-Experimental Design:

A. Descriptive of Causal?

 B. Random Assignment?

C. Static or Manipulated IVs?

D. Control Group?

E. Statistics Used

QUASI-EXPERIMENTAL:

A. Causal

B. Yes & No Random Assignment

C. Both Static and Manipulated IVs

D. Yes & No Control Group

E. Statistics Used:

a. ANOVA

b. ANCOVA

c. MANOVA

d. Regression

1. Multiple Treatment

2. Counterbalanced

3. Crossover

CORRELATIONAL DESIGN Experimental Design:

A. Descriptive of Causal?

B. Random Assignment?

C. Static or Manipulated IVs?

D. Control Group?

E. Statistics Used

CORRELATIONAL DESIGN Experimental Design:

A. N/A

B. Descriptive

C. NO Random Assignment

D. Static IVs

E. NO Control Group

F. Statistics Used:

a. Chi-square

b. Regression

c. Correlation

Name 5 subtypes of an OBSERVATIONAL Design.

1. Case Study

2. Case Control

3. Cross-Sectional

4. Retrospective Cross-Sectional

5. Cohort:

a. Prospective

b. Longitudinal

c. Single Group

d. Multiple Group

e. Accelerated

OBSERVATIONAL DESIGN: Experimental Design

A. Descriptive of Causal?

B. Random Assignment?

C. Static or Manipulated IVs?

D. Control Group?

E. Statistics Used

OBSERVATIONAL DESIGN: Experimental Design

A. Descriptive

B. NO Random Assignment

C. Static IVs

D. NO Control Group

E. Qualitative Statistics

1. ABAB

2. Multiple Baseline

3. Changing Criterion

SINGLE SUBJECT Design: Experimental Design

A. Descriptive of Causal?

B. Random Assignment?

C. Static or Manipulated IVs?

D. Control Group?

E. Statistics Used

SINGLE SUBJECT Design:

A. Descriptive

B. NO Random Assignment

C. Static IVs

D. NO Control Group

E. Statistics Used:

a. Correlation

b. Qualitative

aka "generalizability"

The extent to which the results can be generalized beyond the conditions of the research to other populations, settings, or conditions

aka "confounding effect" "contingency effect" "joint effect" "moderating effect"

• The joint effect of 2+ IVs on a DV

• Occurs when IVs not only have separate effects, but also have combined effects that are different from the simple sum of their separate effects

• Occurs when the relation b/n two variables differ, depending on the value of another variable

• Presence of a statistically significant interaction effect makes it difficult to interpret main effects

**May be ordinal or disordinal

First-order interaction

Second-order interaction

1. 2 variables interact

2. 3 variables interact

**May be ordinal or disordinal

•  Overall or average effects of the variable, obtained by combining the entire set of component experiments involving that factor

• Simple effect of an IV on a DV

• The effect of an IV uninfluenced by (without controlling for the effects of) other variables

• Difficult to interpret Main Effects in the presence of interaction effects

• Used in contrast with the interaction effect of 2+ IVs on a DV

• Attempts to explain, predict, and explore specific relations

• A tentative answer to a RS Q

•  Represent "if-then" statements about a particular phenomenon

-"If” is the IV which is manipulated in some ways

-“Then” is the DV or the resulting data

aka "Research hypothesis"

Hypoth that does NOT conform to the one being tested, usually the opposite of the Null hypoth

Rejecting the Null Hypoth shows that the Altern Hypoth may be true

• Does NOT necessarily refer to 0 or no difference --> Refers to the hypothesis to be nullified or rejected

• Null Hypoth = Core idea in hypothesis testing

• RSer usually hopes to reject Null

• Finding evidence to reject the Null Hypoth increases confidence in the probability that the Altern Hypoth is True

• Problem to be investigated in a study stated in the form of a Q

• Essential for focusing the investigation at all stages, from the gathering through the analysis of evidence

• Usually more exploratory than a RS hypothesis or a Null Hypothesis

• Used to assess the statistical significance of findings

• Involves comparing empirically observed sample findings with theoretically expected findings - expected if the Null hypothesis is true

-This comparison allows one to compute the probability that the observed outcomes could have been due to chance or random error

IF: IV which is manipulated in some way

THEN: The DC or resultant data

Iterative processes (require RSers to continuously run data)

Need to gain support or reject it through multiple studies BC cannot prove models

Null: Not related

Alternative: Related

aka "Beta Error" or "False Negative"

Error made by wrongly retaining (or accepting or failing to reject) a False Null hypothesis (Saying that they aren't related when they are)

aka "Alpha Error" or "False Positive"

• Error made when wrongly rejecting a TRUE Null Hypothesis -- Incorrectly concluding that the variables are related when they are not, or wrongly deciding that a sample statistic exceeds the value that would be expected by chance

• Anything that can be measured or assigned a number, such as unemployment rate, religious affiliation, experimental Tx, etc.

• Opposite of a variable is a constant

aka "predictor variable" or "explanatory variable"

• Manipulated by the experimenter who predicts that the manipulation will have an effect on another variable (the DV)

• Can be used to predict or explain the values of another variable

1. Environmental/Situational: Varying what is done to, with, or by the subject (e.g. a task is provided to some but not to others - Mindfulness Meditation)

2. Instructional: Variations in what participants are told or led to believe through verbal or written statements, usually aimed at altering the perception of the situation (e.g. Different interpretations of tests, based on knowledge about subjects)

3. Organismic: "Static" Cannot be manipulated so subjects canNOT be randomly assigned to these conditions (e.g. Gender, age, year in school)

aka "outcome" "criterion" and "response" variable

• Variable whose values are predicted by the IV, whether or not they are caused by it

• Presumed effect in a study (so called BC it 'depends' on another variable -IV = Presumed Cause

aka "discrete" or "nominal"

Variable that distinguishes among subjects by sorting them into a limited number of categories, indicating type or kind (e.g religion: Christian, Buddhist, Catholic; breaking a continues variable, such as age may also be done)

• Variable that can be expressed by a large (often infinate) number of measures

• A variable that can be measured on an interval or ratio scale

• ALL Continuous variables are interval or ratio, but all Interval or Ratio scales are NOT continous

• Usually used when there are FEW ranks in the data (Ordinal used when there are MANY)

aka "stochastic variable"

• A variable that varies in ways the researcher does NOT control

• Variable whose values are randomly determined -"random" refers to the way the events, values, or subjects are chosen or occur, NOT to the variable itself -e.g. Men and Women are NOT random, but sex COULD be random

-Manipulated

-Observed

-Held constant

1. Statistical Methods of control (ANCOVA)

2. Holding Constant

3. Matching

4. Blocking (Block Design; Randomized-Blocks Design)

5. Counterbalancing

6. Double Blind

7. Control group

a. No Tx Control

b. Wait-List Control

c. Attention Placebo Control

d. Yoked-Control

e. Patched-Up Control

• Provides a way of statistically controlling the linear effects of extraneous variables called "covariates" (variables one does not want to examine in a study)

• Allows you to remove covariates from the list of possible explanations of variance in the DV by using stats (e.g. regression) to partial out of the effects of covariates, rather than direct experimental methods to control extraneous variables (e.g. pretest scores are used as covariates in pre/posttest experimental designs)

• Also used in nonexperimental RS - Surveys of nonrandom samples or quasi-experiments when subjects canNOT be randomly assigned to control and experimental groups

Assign the variable an average value to subtract the effects of a variable from a complex relationship so as to study what the relationship would be if the variable were in fact a constant

-e.g. Education of managers in a study

aka "subject matching"

• RS Design in which subjects are matched on characteristics that might affect their rxn to a Tx

• After pairs are determined, one member of each pair is assigned at random to the group receiving Tx (experimental group); The other group (Control Group) does not receive Tx -

• W/out Random Assignment, matching is not considered good RS practice

• Block Design: Subjects are grouped into categories or "blocks," which are treated as the unit of analysis

-Goal: Control for a covariate

• Randomized-Blocks Design: Subjects are marched on a variable the RSer wishes to control

-Blks are used to ensure each group has subjects with a similar status

-Subjects are put into groups (blocks) the same size as the # of Txs

-Members of each block are assigned randomly to differnt Tx groups

• A group that does NOT receive the Tx so it can be compared to the Experimental Group

• Used to address threats to Internal Validity (e.g. Hx, maturation, selection, testing, etc.)

• Ethical Considerations: problems associated with witholding Tx 

• Types of Control Groups:

1. No-Tx Control Group
2. Wait-List Control
3. Attention-Placebo Control
4. Yoked-Control
5. Patched-up Control

• Like a no-Tx control, but Tx is only withheld temporarily

• Period of time that Tx is withheld usually corresponds to the pre to post test ass't interval --> As soon as the second ass't battery is administered, the waitlist subjects receive their Tx

• BC subjects in wait-list controls receive Tx after the post-test period, long-term followup is not possible since the control is no longer available for comparisons Group Test Tx Test Tx Exper X X X X Control X X X (receives Tx later)

• Used when the procedure changes for each subject based on their performance; Pairs are arbitrarily formed so that the subject in the experimental grp and the yoked-control subject receive same # of sessions/trials/etc.

• Pairs are formed arbitrarily (unless matching was used to assign to grps) so that the subjects in the experimental grp receive & yoked-control grp receive same # of sessions

• Purpose is to ensure that groups are equal with respect to potentially important but conceputally and procedurally irrelevant factors

 Exper Grp Sesh's Control Grp Sesh's Subject1 5 Subject2 5 Subject3 7 Subject4 7 Subject5 3 Subject6 3

A theoretical frequency distribution of the scores for or values of a statistic, such as a mean

ALL stats that can be computed for a sample has a sampling distribution that is the distribution of statistics that WOULD BE produced in a repeated random sampling (with replacement) from the same population

Composed of all possible values of a stat and their probabilities of occurring for a sample of a particular size

Inferential stats depends on sampling distribution

Used to calc the probability that sample stats couild have occurred by chance, and thus to determine whether something that is true of a sample statistic is also likely to be true of a pop parameter

Samp distrib is a distrib used as a model of what would happen if:

a. Null hypoth were true (there really were no effects) AND

b. Experiment was repeated an infinate # of times

-Created using Monte Carlo experiments: A lrg # of equal sized random samples are drawn from a pop you wish to represent

-Stat is computed for each sample and arranged in a freq distrib so you can see the normal curve for that pop -Repeating multiple times --> pop sampling distribution -Any generating of random values in order to study stat models Construction: Assume an infinate # of samples of a given size have been drawn from a pop & distributions recorded; then stat (e.g. mean) is computed for the scores of each hypothetical sample; then arranged in a distribution to arrive at the sampling distribution; sam distrib is compared with the actual sample stat to determine if that stat is or is not likely to be the size it is due to chance

Error that ALWAYS occurs BC a sample is drawn from a pop BC only that part of the pop is measured, leaving out those who are not measured

Error decreases as sample size increases

If pop from which sample is drawn is lrg, pop values will NOT be affected much by 1 or 2 samples who are extreme

• Mean

• Estimated standard deviation of the variable divided by the square root of the sample size

 -Sampling Error = SD/ sqrt(n)

• Not dependent upon the population size, but only on the variability of the variable of concern and sample size

The SD divided by 2 (the square root of four)

Increase the sample size to 16; halve it again by increasing SS to 64

aka "simple random sampling/equal probability sampling" when no qualifiers are used --> Reduces bias

• Principles are the same: Selecting a grp of subjects (a sample) for a study from a larger grp (pop) so that each individual (or other unit of analysis) is chosen entirely by chance

• Random Selection: More often used in experimental RS

• Random Sampling: More often used in survey RS -

• Selecting elements (subjects or other unit of analysis) from a pop in such a way that they are representative of the population

• Purpose: to increase the likelihood of being able to generalize accurately about the population

• Sampling is often a more accurate and efficient way to learn about a lrg pop than a census of a whole population

1. Convenince Sample 
2. Snowball Sample 
3. Stratified Random Sample 
4. Proportional Stratified Random Sample 
5. Cluster Sample 
6. Probability Sample 
7. Quota Sample 
8. Accidental Sample 
9. Purposive Sample

aka "networking sample" "word of mouth"

One subject gives the RSer the name of another subject, who in turn provides the name of a third, and so on -Useful when ppl w/similar experiences are needed

The population as a whole is separated into distinct parts ("strata") and a random or probability sample is drawn from particular categories (or "strata") of the population being studied

• Works best when the indivs within the strata are highly similar to one another and different from indivs in other strata

• Ensures that the full population is properly represented

• Fxn similar to blocks & randomized blks designs -can be proportionate so that the size of the strata corresponds to the size of the grps in the pop (can also be disproportionate)

• Used in political polling

Selecting units (clusters) of individuals rather than individuals themselves, then randomly selecting subjects from those units

A method for drawing a sample from a pop:

1. In 2+ stages

2. When it is not possible to identify or obtain access to entire population

3. When random sample would produce a list of widely scattered subjects and NOT be cost-efficient

•Ex: randomly selecting psyc hospitals and then randomly selecting sx from them

• Want clusters to be as diverse as possible (CONTRASTs stratified sampling - subjs as similar as possible)

• DISADVAN: Each stage of the process increases sampling error --> Margin of error larger in cluster sampling than in simple or stratified random sampling (but may be compensated by increasing sample size BC method is cost-effective)

Each case that could be chosen has a known probability of being included in the sample

ALWAYS uses Random Selection

Often a random sample, which is an equal probability sample

A stratified NONrandom sample (a sample selected by dividing a pop into categories & selecting a certain # [a quota] of respondents from each category)

• Indiv. cases within each category are not selected randomly - usually chosen on basis of convenience

• NOT a reliable method for making inferences about a population

Sample gathered haphazardly (e.g. by interviewing the first 100 ppl you ran into that are willing to talk to you

• NOT a random sample

• DISADVAN: The RSer has no way of knowing what the pop might be

Sample composed of subjects selected deliberately (on purpose) by RSers, usually BC they think certain characteristics are typical or representative of the population

ADVAN:

1. Increase representativeness in RS

DISADVAN:

1. Assumes the RSer knows in advance what the relevant characteristics are
2. May introduce unknown bias
3. NOT random

• Compromise b/n random sampling and purposive sampling = STRATIFIED RANDOM SAMPLING

aka "random allocation"

Individual in each grp is assigned ENTIRELY by chance with equal probability of being placed in each grp 

Goals:

1. Distribute characteristics of a sample among grps (eg age, sex, etc) that, if left uncontrolled, might interfere with interpretation of the grp diffrenences
2. Group Equivalence: (doesn't necessarily happen) - When sample sizes are small, grp equiv is less likely & power of test is attenuated (lowered)

• Uused when a subject variable is known to be related to scores on the DV

• Grp subjects together based on similarity on the variable in question; then randomly assign one person from each pair to the experimental grp

Statistical proposition that the larger a sample size, the more closely a sampling distribution of a statistic will approach a normal distribution (true even if the pop from which the sample is drawn is not normally distributed)

• Usually requires 30+ ppl

• Explains why sampling error is smaller within a larger sample than with a small sample & why a normal distribution can be used to study a wide variety of statistical problems

Another term for DV

• Criterion used in correlational RS when it is not possible to establish a causal relationship (it is like the outcome of the study) b/n a DV and IV

• Measures what it is supposed to measure

• Requires RELIABILITY, but reverse not true

• Extent to which a measure is free of systematic error Refers to designs that help RSers gather data appropriate for answering their Qs

• Problems that can lead to false conclusions

• Characteristics of various RS methods & designs that can lead to spurious or misleading conclusions

• Specific reasons why we can be wrong with inferences about covariance

• Help experimenters anticipate likely criticisms of interences from experiments

• Can reduce threats by using 2+ methods

1. To distribute characteristics of a sample among groups (i.e. age, sex, etc) that, if left uncontrolled, might interfere with interpretation of the group differences

2. Group equivalence (Although doesn't necessarily happen - Small sample size --> Lower Power [Attenuation])

•  Inferences about whether it is reasonable to conclude that covariance exists, given a particular alpha level and given the variances obtained in the study that influence conclusions we reach about an experimental conclusion and its validity

• Concerns 2 related statistical inferences that affect the covariation component of causal inferences:

1. Whether the presumed cause and effect covary

a. Can incorrectly conclude that cause and effect covary when they do not (TI Error) or incorrectly conclude that they do not covary when they do (TII error)

b. Can overestimate or underestimate the magnitude of covariation, as well as the degree of confidence that magnitude estimate warrants

2. How strongly they covary

"TYPE I Error"

The probability of rejecting a hypothesis (the null) when that hypothesis is true

"TYPE II Error"

The probability of accepting the null hypothesis when it is fale

(1 - Beta)

The probability of detecting real differences between conditions

ES = m1-m2/s

• A way of expressing the differences b/n groups, treatments, or conditions

• The magnitude of the difference b/n 2+ conditions expressed in standard deviation units

• Difference b/n the means and the pooled variance -Useful in metaanalysis

**Reasons why inferences about covariation b/n 2 variables may be incorrect

1) Low statistical Power

2)Violated Assumptions of Statistical Tests

3) Fishing and the Error Rate Problem

4) Unreliabilit of measures

5) Restriction of Range

6) Unreliability of Tx Implementation

7) Extraneous variance in the experiemental setting

8) Heterogeneity of units

9) Inaccurage Effect Size Estimation

**Threat to Statistical Conclusion Validity

An insufficiently powered experiment may incorrectly conclude that the relationship b/n tx and outcome is not significant

**Threat to Statistical Conclusion Validity

Violations of statistical test assumptions can lead to either overestimating or underestimating the size and significance of an effect

**Threat to Statistical Conclusion Validity

Repeated tests for significant relationships, if uncorrected for the number of tests, can artifically inflate the statistical significance

**Threat to Statistical Conclusion Validity

Measurement error weakens the relationship b/n two variables and strengthens or weakens the relationships among three or more variables

**Threat to Statistical Conclusion Validity

If a Tx that is intended to be implemented in a standardized manner is implemented only partially for some respondents, effects may be underestimated compared with full implementation

**Threat to Statistical Conclusion Validity

Some features of an experimental setting may inflate error, making detection of an effect more difficult

**Threat to Statistical Conclusion Validity

Increased variability on the outcome variable within conditions increases error variance, making detection of a relationship more difficult

**Threat to Statistical Conclusion Validity

Some statistics systematically overestimate or underestimate the size of an effect

• The extent to which the results of a study can be attributed to the txs, rather than to the flaws in the RS design

• Degree to which one can draw valid conclusions about the causal effects of one variable or another

• Depends on the extent to which extraneous variables have been controlled by the RSer

• Reasons why inferences about the relationship b/n 2 variables may be incorrect

1)Ambiguous Temporal Precedence

2) Selection

3) History

4) Maturation

5) Regression

6) Attrition

7) Testing

8) Instrumentation

9) Additive and Interact effects of threats to Internal Validity

**Threat to Internal Validity

Lack of clarity about which variable occurred first may yield confusion about which variable is the cause and which is the effect

**Threat to Internal Validity

Systematic differences over conditions in respondent characteristics that could also cause the observed effect

**Threat to Internal Validity

Events occurring concurrently with tx could cause the observed effect

**Threat to Internal Validity

Naturally occurring changes over time could be confused with a Tx effect (eg growing older and growing wiser)

**Threat to Internal Validity

When units are selected for their extreme scores, they will often have less extreme scores on other variables, an occurrence that can be confused with a Tx effect

-eg. ppl who come to psychotherapy when they are extremely distressed are likely to be less distressed on subsequent occassions, even if the psychotherapy had no effect ---> Phenomenon called "Regression to the mean"

**Threat to Internal Validity

-aka "mortality"

Loss of respondents to Tx or to meas't can produce artifactual effects if that loss is systematically correlated with conditions

**Threat to Internal Validity

Exposure to a test can affect scores on subsequent exposures to that test, an occurrance that can be confused with a Tx effect

-Practice, familiarity, or other forms of reactivity are relevant mechanisms and could be mistaken for Tx effects

**Threat to Internal Validity

The nature of a measure may change over time or conditions in a way that could eb confused with a Tx effect

-eg. a change in a measuring instrument can occur over time even in the absence of Tx, mimicking a Tx effect (the spring on a bar press might become weaker and easier to push over time artifactually increasing reaction times)

• The impact of a threat can be added to that of another threat or may depend on the level of another threat

• Validity threats do NOT operate singly -->several can operate simultaneously: Net bias depends on the direction and magnitude of each individual bias plus whether they combine additively or multiplicatively (interactively)

-eg. Selection-maturation additive effect may result when nonequivalent experimental groups formed at the start of the Tx are also maturing at different rates over time

-A selection-hx additive effect may result if nonequivalent groups also come form different settings and each group experiences a unique local hx

 Extent to which the RS Design or strategy allows for a clear interpretation of the basis of the relationship among variables of interest

• Convergent and Discriminant validity: Used as tests of Construct Validity

1. Reasons why inferences about the constructs that characterize study operations may be incorrect

2. Sources of secondary variance

3. Features within an experiment that interfere with interpretation and create alternative explanations for results which are different from the theoretical assumptions about the action of the IV or the presumed agents of change

1. Inadequate Explication of Constructs

2. Construct Confounding

3. Attention and/or Simple Contact with Participants (Hawthorne Effect)

4. SIngle Operations and Narrow Stimulus Sampling

5. Experimenter Expectancy Effects

6. Demand Characteristics or Cues in the Experimental setting

**Threat to Construct Validity

Failure to adequately explicate a construct may lead to incorrect inferences about the relationship b/n operation and construct

**Threat to Construct Validity

Operations usually involve more than one construct; Failure to describe all the constructs may result in incomplete construct inferences

**Threat to Construct Validity

aka "Hawthorne Effect"

The intervention may impact participatnts simply because of the attention or human contact it provided, rather than the specific content or characteristics of the intervention

• Controlled for by the use of Attention Control --> Giving the group attention but not the intervention

**Threat to Construct Validity

The way in which the intervention, tx, or program is operationalized or delivered may limit the RSer's ability to examoine why the intervention affected the participants

-eg. Holding constant on the therapist - but the therapist might be more adept or confident with one intervention than itself --> Therefore the results are due to the therapist, not the Tx

**Threat to Construct Validity

The extent to which it is possible that the RS'ers beliefs, ideas, hopes, opinions, and hypotheses inadvertently affected participants responses

• May be communicated either verbally or nonverbally

• Controlled for use by the blind and double-blind studies

**Threat to Construct Validity

The extent to which extraneous cues in the intervention or experimental procedure may account for the results, rather than the intervention itself

aka "generalizability"

The extent to which the findings of a study are relevant to subjects and settings beyong those in the study

Reasons why interences about how study results would hold over variations in persons, settings, txs, and outcomes may be incorrect

1. Inadequate explication of constructs

2. Construct confounding

3. Sample characteristics

4. Stimulus characteristics and settings

5. Reactivity of experimental arrangement

6. Multiple txs

7. Novelty Effects

8. Experimenter Expectancy

9. Reactivity of Ass't

10. Test Sensitization

11. Timing of Measurement

**Threat to External Validity

Failure to adequately explicate a construct may lead to incorrect inferences about the relationship b/n the operation and construct

**Threat to External Validity

Operations usually involve more than one construct, and failure to describe all the constructs may result in incomplete construct inferences

**Threat to External Validity

The degree to which the results of the research may be generalized to others who vary from the particular characteristics of the selected sample

-eg. Demographics, age, gender, religion, ethnicity, nationality, ability, or disability, SES, or geography

**Threat to External Validity

The degree to which the conditions in the natural RS setting may impact the results in ways which may not generalize to situations or persons who are not involved in an experiment

**Threat to External Validity

Participant awareness that they are participating in an experiment may impact the results in ways which may not generalize to situations persons who are not involved in an experiment

**Threat to External Validity

When participants receive more than one experimental condition or tx, the results may not generalize to situations where only a single tx is given

**Threat to External Validity

The possibility that effects of an intervention are in part due to the novelty of the administration circumstances

**Threat to External Validity

The unintentional effect that the experimenter exerts on the study in the direction of the hypothesis

**Threat to External Validity

Participants may respond to tests or ass't measures differently when they are aware that they are being assessed, than if they are not aware --> Participants may not be aware of such meas't or ass't in other non-experimental situations --> the results would perhaps not be generalizable

**Threat to External Validity

• The effect produced by pretesting participants

• May make them more or less attentive or receptive to the intervention and limit generalizability

**Threat to External Validity

The degree to which the results of the intervention or RS project vary as a result of the point in time which the post-intervention is given

aka "pre-experimental design"

• Descriptive (Qualitative)
• NO random assignment
• Poor External Validity/Generalizability
• Investigators observe subjects, but do NOT interact with them

*SUBTYPES:

1. Case Study

2. Case Control

3. Cross-Sectional

4. Retrospective Cross Sectional

5. Cohort Design : Used to collect preliminary information that may lead to specific IVs, DVs, and hypotheses about relationships

**Observational Design

An in depth study of a single individual, organization, family, or other social unit

ADVAN: Allows more intensive analysis of specific empirical details

DISADVAN:

1. Hard to use results to generalize to other cases

2. Purely descriptive

**Observational Design

Method of sampling cases with and without that outcome and studying their backgrounds

-eg. In a study of lung cancer, the cases are individuals who have the disease; the controls are similar people who do not have it; The background of those with and without the disease are compared to understand the origins of the disease

**Observational Design

A study conducted at a single point in time by taking a "slice" (cross section) of a population at a particular time

INDIRECT evidence about the effects of time ONLY

Caution when drawing conclusions about change (eg. Just BC older age group is more prejudiced than a younger age group does NOT mean that the younger grp will become more prejudiced as they grow older

**Observational Design

Draw inferences about an antecedent event that results in/is associated with an outcome

Observe the past to see what determined present-day outcomes

Attempts to ID the timeline b/n possible cause or antecedents (risk factors) and subsequent outcome of interest

Subjects ID'd who already show the outcome of interest (cases) and compared to those who do not show the outcome (control)

-eg. Rel. of attachment patterns to suicidal bx among adolescents

**Observational Design

aka "Cohort analysis" or "Prospective Longitudinal Study"

Study same cohort over time - Strength of design lies in establishing the relations b/n antecedent events and outcomes

Cohort: Group of indiv having a statistical factor (usually age) in common

PROBLEM: May confound results of cross-sectional as diff. are more due to the specific cohort and not the DV you are interested in

Difference from Case-Control: Cohort designs follow sample over time to ID factors leading to an outcome of interest and the grp is assessed before the outcome has occurred

a. Single Cohort: One grp who meets particular criteria are selected are followed over time in order to study the emergence of the outcome

b. Multiple-Group cohort: 2+ groups are ID'd and followed over time to examine outcomes of interest

c. Accelerated Cohort: The study of multiple grps - cohorts who vary in age are included - ea grp covers only a portion of the total time frame of interest and the grps overlap in ways tha allow the investigator to discuss the entire developmental period; b/n longitudinal and cross-sec desigs; Requires less time than if a single grp were studied -

REQUIREMENTS:

1. Random Assignment

2. IVs must be Manipulated

a. Manipulated: Differences chosen by researcher

b. Static: RSer cannot assign to subjects (age, gender, sex orient)

3. Control Group: Grp that experiences all the things as the experimental grp but does not receive tx (eg. Hx, Maturation, demographics, selection, testing, etc); Accounts for spontaneous remission cases which affect internal validity

STRENGTH: Internal validity (due to Random Ass't) - more certain about attributing cause to the IV -->Greatest control of IVs and Error

WEAKNESS: External Validity - may be inappropriate to generalize results beyond lab

1. Pretest-Posttest Control Group Design

2. Posttest Only Control Group Design

3. Solmon 4 Group

4. Factorial Design

• 2 levels of the IV with 1 receiving Tx and the other not receiving tx

• Assesses amt of change within a group from time 1 (pretest) to time 2 (posttest)

• DOES NOT control for pretest sensitization RA Observation Tx Pretest R O X O R O O

• At least 2 levels of IV with 1 receiving Tx

• Only a postest is given for pretest sensitization, however we cannot assure that our groups started out equivalent

RA Observ Tx Postttest R X O R O

• Used to control the effects of pretesting by including a pretest IV

• Costly - Requires a great amt effort, and a lrg number of subjects is needed!

Group RA Observ Tx Posttest 1 R O X O 2 R O O 3 R X O 4 R O

• **G2 v. G4: Assesses the effect of pretest on internal validity (neither grp got Tx)

• G1 v. G3: Assesses effect of pretest on external validity (both grps got Tx)

• Within subjects design in which the DV is administered to all subjects before and after the IV is applied (pre, post, f/u)

• Focuses on comparing subjects against themselves which increases power by decreasing error variance RA Observation Posttest F/U R O O O R O O O

• Involves 2+ IVs, each with 2+ levels

• Provides info on:

a. MAIN EFFECTS of each IV

b. INTERACTION b/n 2 variables

RA Obser Tx F/U R Xa1b1 O R Xa1b2 O R Xa2b1 O R Xa2b2 O

• STRENGTHS:

1. It can assess the effects of separate variables

2. Different variables can be studied with fewer subjects

3. Provides unique info about the combined effects of IVs

4. Ixns provide important info such as whether there may be variables that moderate the effects of other variables

WEAKNESSES:

1. # of grps multiplies quickly as new factors or new levels are added

2. Optimally informative when an investigator predicts an interactive relationship among variables, but with more variables, interactions become complicated and difficult to interpret

aka "Combo design"

RERQUIREMENTS:

1. At least one IV manipulatable

2. One IV is static: True Random Assignment is NOT possible

3. Must meet req'ts for causal relationships:

a. Cause precede effect

b. Cause covaries with effect

c. Alternative explanations for causal rel. are implausible

Increase statistical power by controlling individual differences b/n units within conditions; Use fewer units to test the same # of Tx 

--> DRAWBACKS:

1. Fatigue effects

2. Practice effects

3. Carryover effects

4. Order effects

SOLUTION: Counterbalancing: Some units get Tx in on order, and others get it in another order so that order effects can be assessed

aka "Sequence effects" "Multiple Tx interference" "Carryover Effects"

The influence of order on subject responses in which subjects receive Tx (within-subject design)

PROBLEM: Confound Tx effects

SOLUTION: Counterbalancing -eg. Survey: Order of Qs

Designs that try to balance the order of Txs across subjects

In a within-in subjects factorial experiment, presenting conditions (Txs) in all possible orders to avoid order effects

eg. Latin Square

Partway through the experiment (usually midway), all subjects cross over (are switched) to another experimental condition

• Both groups get both control and experimental conditions

• Increases POWER BC each group serves as its own control

Controls for carryover (order) effects that may result with a within-subjects design

Administer levels of an IV to different subjects or groups in a different order (Balances the order of Txs)

A method of allocating subjects, in a within-subjects experiment, to Tx group orders -

• GOAL: Avoid order effects by rotating the order of Tx

• Number of rows and columns MUST be equal

Describes what happens when many subjects in a study have scores on a variable that are at or near the possible upper limit

• Makes analysis difficult BC it reduces the amount of variation in a variable

Describes a situation in which many subjects in a study measure at or near the possible lower limit

• Makes analysis difficult BC it reduces the amount of variation in the variable

aka "Single case design"

Compare the effects of different Tx conditions on performance of one individual over time

REQUIREMENTS:

1. Baseline Assessment:

a. Observe Bx for a period of time prior to the intervention to predict the level of performance

b. Data must be stable (absence of a slope)

2. Continuous Ass't: Performance is observed on several occasions prior to the intervention, then continuously observed during the period of time the intervention is in effect

3. Examination of Trends/Slpoe: Tendency for performance to decrease or increase systematically or consistently over time

Single-subject Design/Within-subject design (can be an indiv., family, organization, community, group)

CONTRAST TO GROUP EXPERIMENTAL DESIGN: B/n subject designs (Participant is either in the Tx or control grp)

1. ABAB

2. Multiple Baseline

3. Changing Criterion

aka "Withdrawal" or "Reversal Design"

Single-Subject Design

• Alternate baseline measures of a variable with measures of that variable after a Tx

• One Tx

• NO Control grp -"A": Baseline (Control); "B": Tx (Intervention)

• ABAB used when unethical to withold a Tx from a control group

• Extraneous EVENTS are much better controlled when there are several shifts b/n baseline and intervention phases

• DRAWBACKS:

1. Carry-over Effects

2. Order Effects

3. Irreversibility of Effects

4. Ethical Problems

5. Feasibility Problems

Demonstrates effects of an intervention by showing that bx change accompanies the intro of the intervention at different pts in time

• Baseline period of ass't is taken, and then the intervention is introduced to the different baselines at different points in time

• Data collected continuously by taking ass'ts over time of at least 2 precisely defined bxs

• Main Difference from ABAB Designs: At least 2 bxs are studied concurrently

Demonstrates the effect of an intervention by showing that bx changes incrementally to match a performance criterion (expected goal/ourcome)

A --> B --> Criterion Reached and Reward Administered --> Set New Goal, Return to A --> B --> Goal reached and Reward Given...

Both are longitudinal and concerned with:

1. Issues of control

2. Specifying targets of intervention in operational terms

3. Developing measurement and recording plans for assessing these targets

4. Can use a combination of procress and outcome measures, though single subject designs rarely employ process measures, which try to assess the "black box" of Tx or what kinds of interactions go on b/n clients and therapists during the course of an intervention

1. Single-Subject designs typically use more repeated measures

2. Duration of RS is more variable

3. Participants are more actively involved in setting the goals and targets of interventions

4. The choice of the design is typically established by the worker rather than the RSer

5. Designs are more flexible, responding to the needs of the particular case rather than fixed

6. Findings have more direct and immediate impact on interventions at the individual case level

7. Less costly than group

Designs conducted to get an overview or a review of all literature in a specific area through the evaluation and combination of results from multiple studies

• GOAL: Provide estimate of the Effect Size for the particular area of RS

Psychologists provide accurate info about their RS proposals and obtain approval prior to conducting the RS

• Conduct RS in accordance with the approved RS protocol

Psychologists inform participants about:

1. The purpose of the RS, expected duration, and procedures

2. Their right to decline to participate and to withdraw from the RS once participation has begun

3. The foreseeable consequences of declining or withdrawing

4. Reasonably foreseeable factors that may be expected to influence their willingness to participate such as potential risks, discomfort, or adverse effects

5. Any prospective RS benefits

6. Limits of confidentiality

7. Incentives for participation

8. Whom to contact for questions about the RS and RS participants' rights

1. The experimental nature of the Tx

2. Services that will or will not be available to the control group(s) if appropriate

3. Means by which assignment to Tx and control grps will be made

4. Available Tx alternatives if an indiv does not wish to participate in the RS or wishes to withdraw once a study has begun

5. Compensation for or monetary costs of participating including, if appropriate, whether reimbursement from the participant or a third-party payor will be sought

Psychologists obtain informed consent from RS participants prior to recording their voices or images for data collection unless:

1. The RS consists solely of naturalistic observations in public places, and it is not anticipated that the recording will be used in a manner that could cause personal identification or harm

2. RS design includes deception, and the consent for the use of the recording is obtained during debriefing

Must take steps to protect the prospective participants form adverse consequences of declining or withdrawing from participation

• When RS participation is a course requirement or opportunity for Extra Credit, the prospective participant is given the choice of equitable alternative activities

Psychologists may dispense with informed consent only:

1. Where RS would not reasonably be assumed to create distress or harm and involves:

a. The study of normal educational practices, curricula, or classroom management methods conducted in educational settings

b. Only anonymous questionnaires, naturalistic observations, or archival RS for which disclosure of responses would not place participants at risk for criminal or civil liability or damage their financial standing, employability, or reputation, and confidentiality is protected

c. The study of factors related to job or organization effectiveness conducted in organizational settings for which there is no risk to particiants' employability, and confidentiality is protected

2. Where otherwise permitted by law or federal or institutional regulations

Psychologists make reasonable efforts to avoid offering excessive or inappropriate financial or other inducements for RS participation when such inducements are likely to coerce participation

• When offering professional services as an inducement for RS participation, psychologists clarify the nature of the services, as well as the risks, obligations, and limitations

1. Psychologists do NOT conduct a study involving deception unless they have determined that the use of deceptive techniques is justified by the study's significant prospective scientific, educational, or applied value and that effective nondeceptive alternative procedures are not feasible

2. Psychologists do NOT deceive prospective participants about RS that is reasonable expected to cause physical pain or severe emotional distress

3. Psychologists explain any deception that is an integral feature of the design and conduct of an experiment to participants as early as is feasible, preferably at the conclusion of the data collection, and permit participants to withdraw their data

Psychologists provide a prompt opportunity for participants to obtain appropriate info about the nature, results, and conclusions of the RS, and they take reasonable steps to correct any misconceptions that participants may have of which the psychologists are aware 

• If scientific or human values justify delaying or withholding this info, psychologists take reasonable measures to reduce the risk of harm

Psychologists provide a prompt opportunity for participants to obtain appropriate info about the nature, results, and conclusions of the RS, and they take reasonable steps to correct any misconceptions that participants may have of which the psychologists are aware

• If scientific or human values justify delaying or withholding this info, psychologists take reasonable measures to reduce the risk of harm

• When psychologists become aware that RS procedures have harmed a participant, they take reasonable steps to minimize the harm

Psychologists do not fabricate data

• If psychologists discover significant errors in their published data, they take reasonable steps to correct such errors in a correction, retraction, erratum, or other appropriate publication means

Psychologists do NOT publish, as original data, data that have previously been published

• Does NOT preclude republishing data when they are accompanied by proper acknowledgment

After RS results are published, psychologists do not withhold the data on which their conclusions are based from other competent professionals who seek to verify the substantive claims through reanalysis and who intend to use such data only for that purpose, provided that the confidentiality of the participants can be protected and unless legal rights concerning proprietary data preclude their release

• Does NOT preclude psychologists from requiring that such individuals or grps be responsible for costs associated with the provision of such info

• Psychologists who request data from other psychologists to verify the substantive claims through reanalysis may use shared data only for the declared purpose

• Requesting psychologists obtain prior written agreement for all other uses of the data

Used to organize and describe the characteristics of a collection of data

1. Measures of Central Tendency (Mean, Median, Mode) 2. Measures of dispersion or variability (Range, Standard Deviation, Variance, Confidence Interval)

Grps of data can be summarized using averages

• Mean, Median, Mode (each provides a different type of info about a distribution of scores and is simple to compute and interpret)

-aka "Typical" "average" "Most Central Score"

• Measure of Central Tendency

• Sum of all values in a grp divided by the # of values in the grp

• Sometimes represented by the letter M

Midpoint in a set of scores: 50% fall above, 50% fall below

• Measure of Central Tendency

• An average

• No standard formula for computing: List values in order and find most middle score (if even, take average of 2 most middle values)

• NOT affected by outliers

 Most frequently occurring score

• Measure of Central Tendency

• List values in a distribution, tally # of times that values occurs

• Difference b/n the minimum and maximum score Measure of dispersion/variability

• Purpose: get a general estimate of how wide or different scores are from another --> how much spread in a distribution

• Most general measure of variablility: Subtract lowest score from highest

• Should NOT be used to reach any conclusions about how individual scores differ from one another

Deviation from the standard: Average amt of variability in a set of scores/Distance from the mean -

SD = sqrt ((Sum of (x - Mean) squared)/(n-1))

 Square Root of variance

• Larger SD = more spread across distribution/more different from one another & mean of distribution

• Sensitive to extreme scores

The amount which scores vary or are different from each other & from the group mean

• Variance squared = SD

• If you know the SD of a set of scores and you can square a #, you can easily compute the variance of the same set of scores squared = (sum of (x-mean)squared)/(n-1)

1. Both are measures of variability, dispersion, or spread

2. Formulas are similar (difference is the sqrt - Variance squared = SD)

SD: (sqrt of average summed squared deviation) is stated in original units from which it was derived

Variance: stated in units squared (sqrt is never taken)

An estimated range of values which is likely to include an unknown population parameter

• Width gives idea about how uncertain we are about the unknown parameter --> The smaller the range, the better the estimate

• -eg. in IQ testing, you get a confidence interval for each subscale

• Approx 68% of the scores in a normal distribution are b/n the mean and +/-1 SD 95% of the scores are b/n the mean and +/-2 SD 99% of the scores are b/n the mean +/-3 SD

A desired percentage of the scores (often 95% or 99%) that would fall within a certain range of confidence limits

• Calculated by subtracting the alpha level from 1 and multiplying the result times 100 (eg 100 x (1-.05) = 95%

Lower and upper boundaries / values of a confidence interval and define the range of a confidence interval

Upper and lower bounds of a 95% CI are 95% confidence limits

Specifies a range of values within which the unknown population parameter (the mean) may lie

The (2 sided) CI for a mean contains all the values of 0 (the true population mean) which would not be rejected in the 2-sided hypothesis test if: H0: μ = μ0 against H1: μ not equal to μ0

Width of the CI gives us some idea about how uncertain population parameter (in this case, the mean)

Specifies a range of values w/in which the difference b/n the means of the two populations may lie

The confidence interval for the difference between two means contains all the values of μ1 - μ2 (the difference between the two population means) which would not be rejected in the two-sided hypothesis test of:

H0: μ1 = μ2 against H1: μ1 not equal to μ2

i.e. H0: μ1 - μ2 = 0 against H1: μ1 - μ2 not equal to 0 -Compare one sample t-test

There is no significant difference b/n the means of the 2 populations at a given level of confidence

68% of the curve includes +/- 1 standard deviation

95% of the curve includes +/- 2 standard deviations

99% of the curve includes +/- 3 standard deviations

So you have an end value that is more generalizable to the population

-eg. df = n - 1

If you get a z score of +1, then you are 1 SD above the mean IQ = mean is 100 SD = 15 SD of +1 is a z score of 1 which is equal to a score of 115

Large variance means ppl's scores are difficult to predict

These provide you with the best score for:

1. Describing a group of data 

2. A measure of how diverse/different scores are from another

Descriptive Statistics

1. Central Tendency

2. Variability

Measure of the lack of symmetry of a distribution

Positively Skewed: Points in the (+) direction --> mean > median > mode (mean pulled in direction of extreme scores)

Negatively Skewed: Points in the (-) direction --> Mean < Median < Mode

How flat or peaked a distribution appears

1. LEPTOKURTOTIC: Extent to which the data is concentrated in the middle or at the tails of a distribution --> Majority of scores in the middle

2. PLATYKURTONIC: Looks flat BC scores are evenly spread out through the middle and tails; Probability of responding the same; Uniform distribution

3. BIMODAL: Mean and median fall at the same point; 2 Modes correspond to the two highest points in the distribution

4. MESOKURTOTIC (NORMAL): Mean = Median = Mode

Represented by a bell-shaped curve with most of the scores gathering in the middle and a few extreme scores pulling the tails out a bit

1. Nominal Variable

-Nominal Scale (level of meas't)

2. Discrete Variable

3. Ordinal variable

-Ordinal scale (level of meas't)

4. Interval scale (level of meas't)

5. Ratio scale (level of meas't)

aka categorical/discrete/qualitative variable

NOMINAL SCALE: Numbers stand for names, but have no order value

• Breaking a continuous variable by sorting them into a limited number of categories, indicating type or kind

-e.g. coding female = 1 and male = 2 would be a nominal scale

A way of measuring that ranks subjects (puts them in order) on some variable

• Differences b/n the ranks need not be equal (as they are in an interval scale)

-eg. Team standings; scores on an attitude scale, shirt sizes of small, medium, lrg, xtra lrg

Describes variables in such a way that the distance b/n any 2 adjacent units of meas't (or "intervals") is the same

• NO meaningful 0 point

• Scores can be added and subtracted, but NOT multiplied or divided

-eg. Fahrenheit Temperature scale: NO true 0 point, Equal distance b/n each #

aka "Rank Order"

Any 2 adjoining values are the same distance apart

• There IS a true 0 point

-eg. Height: same distance b/n 70 & 71" & 20 & 21";l 70" is twice the size of 35"

• CanNOt be made about measures on an ordinal scale (eg. the 4th tallest person is twice as tall as the 2nd tallest person)

• Measure of relative location in a distribution

• Most commonly used standard score

• In SD units: Gives the distance from the mean of a particular score

Mean = 0

SD = 1

eg. z-score of 1.25 = one and a quarter SD above the mean

• Useful for measuring performance (eg. on tests)

TO FIND Z: Take your score, subtract from it the mean of all the scores, and divide the result by the SD.

z = (x - μ)/ o or z = (x - M)/SD X is your score, M is the mean, and SD is the standard deviation

It eliminates decimals and negative numbers

• DIFFERENT THAN A t-test

• A standard score in which the mean of the distribution = 50 and the SD = 10

• Z-score is obtained by transforming the z-score (multiplying z by 10 and adding 50), which is why the the z-score is called a transformed standard score

aka "Distribution free stats"

Stat techniques for data that is NOT normally distributed

• Measurable with nominal or ordinal scales

• Allows analyzation of data that come from frequencies (eg. # of kids in different grades; % of ppl receiving social security)

• Less power than parametric tests

Stat techniques for data that approximate a normal distribution

• Measurable with interval or ratio scales

-"H"

• Nonparametric, one-way ANOVA for rank order data

• Based on MEDIANS, not means

• Nonparametric test of significance used when testing >2 Independent samples

• Extension of Mann-Whitney U test & of the Wilcoxon test to 3+ Independent samples

Nonparametric test of statistical significance for use with ordinal data from correlated groups

• Nonparametric version of a one-way, repeated measures ANOVA

• Similar to Wilcoxon test, but can be used with more than 2 groups

• Extension of the sign test

Test of the statistical significance for rank order data of differences between 2 independent grps

• Nonparametric equivalent of the t-test

• Assess whether or not the ranks of observations in one grp is the same as the ranks of observations in another group

• Similar to the Wilcoxon test

aka Wilcoxon “signed-rank” or “rank-sum” test for ordinal data

Nonparametric test of statistical significance for use with 2 correlated samples and the data are rank ordered, such as the same subjects on a before-and-after measure

Kolmogorov Test

Kolmogorv Smirnov Test

• Nonparametric tests (for ordinal data) with 1 grp

• Assess probability that the distribution of sample observations is likely, given hypothesized sample distribution

K-S

• Used with ordinal data for studies that involve 2 groups

• Assesses the probability that the distribution of ordered observations of one group is the same as the other group

Univariate Analysis: Studying the distribution of cases of one variable only (eg. studying the ages of welfare recipiants, but not relating that variable to theis sex, ethnicity, etc.)

• Occasionally used in regression analysis to mean a prob in which there is only one dep variable:

1. Sampling Distributions:

2. Assumptions and their violations:

a. Indep of Observations

b. Normality of distribution

c. Homogeneity of Variance

**Univariate Parametric Statistics

• Exist for each particular statistic: Mean, variance, correlation coefficient, F test, etc

• Each has a standard error

• Created with Monte Carlo study:

1. A lrg # of equal sized random samples are drawn from a population you wish to represent

2. Stat is computed for each sample

3. Stats are arranged on a freq distribution to see a normal curve --> Done repeatedly gives you the population sampling distribution

1. Bell-shaped theoretical distribution

2. Mean, median, mode are the same

3. Z-scores are used as the standard deviation unit

4. Tails of the curve are asymptotic (they never cross the x axis)

5. 68%, 95%, 99%

• Distribution should be Mesokurtotic

• Violations: -/+ skewed, leptokurtotic, platykurtotic, or bimodal

• ANOVAs are robust (resistant) when grps have equal sample sizes to violations of normality --> not a big deal

• But, does NOT hold for impalanced design (eg. n=20 for group 1, n=50 for group 2)

• For more conservative test, use nonparametric test (Kruskal-Wallis, Mann Whitney U, Wilcoxon rank sum test) instead of ANOVA

Variance w/in each condition/grp is similar to the other grps

• Within grp and b/n grp variances are similar

• Tests for this assumptions:

a. Levene: Assess the diff of scores from each of their MEANS

b. Brown Forsythe: Assess diff of scores from each of their MEDIANS

c. F Max: Biggest variance/smallest variance (if biggest is 4-10x greater, then you have a violation) -look up pg46 for ex

1. Levene: Asseses diferences of scores from each of their MEAN

2. Brown Forsythe: Asseses difference of scores from each of their MEDIANS

3. F Maxm(if biggest variance is 4-10x greater = Major violation)

Used to test for significance b/n the means of 2 grp

• Used with Interval or Ration data ONLY

• Either 1 or 2 tailed

-1 tailed: Directional BC tests the hypoth that the mean of 1 or the 2 grp averages is bigger

-2 tailed: Test significance of a "nondirection" hypothesis (a hypoth that says there is a difference b/n 2 averages w/out saying which of the 2 is bigger 

**If RSer is uncertain which is larger, the 2-tailed test should be used

• T test is a special case of the F test:

t(squared) = f, f = sqrt of t

If t is significant, F is significant

?Also used as a test statistic for correlation and regression coefficients

• The one you use depends on the nature of the data and the grp being studied, usually whether the grps are indep or correlated

• Basic Design

a. One IV

b. One DV

c. Stats: t (t2 = F)

• Stats Options:

1. Single sample

2. Independent Sample

3. Paired (related)

**Assumption: Amt of variability in each grp is equal --> May be violated if sample size is big enough

• Small samples & violation of assumption may = Ambiguous results and conclusions

WHAT KIND OF TEST IS THIS AN EXAMPLE OF?

Researchers are interested in comparing the number of antibodies following an influenza shot in the corporate world. Corporate employees were randomized to receive Mindfulness Based Stress Reduction therapy or Music therapy. They were all given a flu shot and antibody levels were tested 3 months later.

300 participants are given a questionnaire to assess their level of emotional impact from 9-11. The scores for fireman were compared with the scores from policeman participants.

WHAT IS THIS AN EXAMPLE OF?

Students are compared before they finish their statistics project and after finish their project on depression by the BDI (Beck Depression Inventory).

You are investigating the sleep efficiency of women diagnosed with nonmetastatic breast cancer. You assign a sleep efficiency score before and after a sleep hygiene class of 4 weeks.

How sensitive a test is to violation

Very Robust = NOT sensitive = NOT biasing the data

1. Normality of distribution

2. Independence of sampling

3. Homogeneity of variance

ANOVA is NOT very sensitive to violation of normality -ANOVA is more robust to normality

• Test for a violation by eyeball - plot with histograms:

If UNIVARIATE NORMALITY: Use Z scores & charts

If BIVARIATE NORMALITY: Check through Scatter plot matrix - look for elliptical shapes

If MULTI-VARIATE NORMALITY: Assess through Mahlinobis - check for outliers

• How to Fix data:

1. Tranform the data

b. Use nonparametric test

**Assumption of a t-test

• ANOVA is VERY sensitive to violations of independent sampling

•  A violation of independence:

 1. Design of experiment

2. Lack of random sampling

• Can NOT fix through POST-HOC
• Does NOT improve with increasing your sample size

**Assumption of ANVOVA, ANCOVA

• It IS sensitive to violations of homogeneity
• You CAN fix a violation through transformations
• Test of the statistical signifiance of the differences among the mean scores of 2+ grps on 1+ variables or factors
• Extension of a t-test (which can handle only 2 grps at a time) to a larger # of grps
• Used for assessing the statistical significance of the rel b/n categorical IV and a continuous dependent variable
• PROCEDURE (in ANOVA): Compute a ration (F ratio) of the variance b/n the grps (explained variance) to the variance within the grps (Error Variance)

1. Source of the variance

2. Explained Variance

3. Error Variance/Unexplained Variance

4. Sum of squares (Total of squared deviation scores)

5. Degrees of Freedom

6. Mean Squares: Calculated - SS/Df

7. F: Ratio of MS b/n to the MS w/in

• Logic: Assesses the differences b/n grp means; Involves partitioning the total variance into components (e.g. within grp and b/n grp)

• Partitioning the Variance: Dividing up the different sources of variance into sums of squares

• Sums of Squares: A mz of variability around the mean of the distribution (SST = SSbg +SSwg)

• Sum of Squares converted into Mean Squares:

MST = SST/df

MSB = SSB/df

MSW = SSW/df

• Mean Square BG:

-An estimate of the variance b/n Tx grps

-Reflects the variability due to differences b/n grp means

-Grand mean: Represents the variability due to error + the effects of the IV

• Mean Square WG (Error Term):

-An estimate of the pooled variance within Tx grps

-Reflects the variability among subjects that are treated alike

-Only reflects variability attributed to error

• Omnibus F = Overall F
• F = MSB/MSW
• F = (Error + Tx)/Error

ANOVA formula (ratio) compares the amt of variability b/n the groups (which is due to chance)

• If Ratio = 1, the amt of variability due to W/in grp diff = amt of variability due to b/n grp differences

--> No sig of diff b/n the grps

• As average diff b/n grps gets larger (Numerator increases in value), F value increases

--> As F value increases, it becomes more extreme in relation to the distribution of all F values and is more likely due to something other than chance

F = t2

(F value for 2 grps = t value for 2 grps squared)

• t values (always used for the test b/n the difference of the means for 2 grps) and an F value (which is always more than two groups) might be related

What are the assumptions of ANOVA?

What tests can you use to test for HOV?

What tests do you use to correct for this violatioin?

1. Independence of scores:

-Each observation is NOT related to the other

-Achieved through RA

-NO way to correct for violation of this

2. Normal Distribution (normality):

-Population is normally distributed

3. Homogeneity of Variance:

-Inflates Type I Error

4. F max

-Test for HOV

-Largest variance/Smallest variance

-Want <9

5. Test for HOV:

-Brown-Forsyth

-Bartlett test

-Hartly

-Cochran

6. Correction for this violation:

-Welch W test

-Brown-Forsyth F* test

2+ IVs (2-way, 3-way., etc.)

• SSB = SS(A) + SS(B) +SS(AxB)

-SSA = Sum of squares for Factor A

-SSB = Sum of squares for Factor B

=SSA x B = Sum of squares for AxB interaction

• 3 x 2: 3 levels of one grouping factor, 3 levels of another

 --> 6 different possibilities: X Axis = IV1 -Separate Lines = IV2; Y Axis = DV

An interaction is present when:

1. Simple Effects of one IV are NOT the same at all levels of the second IV

2. When one of the IVs does not have a constant effect at all levels of the other IV

• Ordinal: Lines are parallel
• Disordinal: Lines intersect or are not parallel

When you are looking for difference b/n 2+ grps

• ONLY data you can use: 1. Interval 2. Ratio

• 1 IV: One way ANOVA

• 2 IVs: Factorial ANOVA

• A within grp variable = Repeated Measure

Subjects participate in EVERY condition

-eg. Use pretest, posttest, F/U: Effects of yoga on sleep IV#1 = Tx (yoga v. psychoeducation) IV#2 = Time (pretx v. posttx v. F/u) RM DV = sleep inventory score

**2 WAY ANOVA w/1 RM

Group differences by comparing the means of each group

Involves spreading out the variance into different sources

Measures variability

• Sum of the squared deviation of scores from the mean

-SS1: Sum of squared deviations of group means from grand mean for IV1

-SS2: Sum of squared deviations of group means from the grand mean (IV2) SS (1x2): Sum of Squares for the intx b/n IV1 & IV2

-Result of adding together the squares of deviation scores

To Calculate:

1. Subtract average of scores from each score

2. Square each answer

3. Add up answers B/N GRPS SS: Sum of squared deviations of GROUPS MEAN from GRAND MEAN

-WITHIN GRPS SS: Sum of squared deviations of INDIVIDUAL SCORES from GROUP MEAN

-TOTAL SS: SS b/n + SS w/in

A measure of b/n grp differences:

Used to compare within-grp differences to compute the F ratio in an ANOVA

To Calculate:

1. Square the deviation scores (each score divided by the mean)

2. Add them up

Define: 

1. Factorial Design

2. Simple Effects of IV

3. Interaction of IV

4. Main effect of IV

1. Factorial Design: Consists of a set of single-factor designs in which the same IV is manipulated but in combination with a second IV

2. Simple Effects of IV: Difference associated with the single-factor experiment involving factor A at level b1

3. Interaction of IV: Present when we find that the simple effects associated with one IV are not the same at all levels of the other IV

4. Main Effects: Overall or average effects of the variable, obtained by combining the entire set of component experiments involving that factor

a = grps; n = subjects

B/n Grps: a-1

W/in Grps: a(n-1)

"purer" form of SS

B/n Grps: SS/Df ----> "Tx Effect"

W/in Grps: SS/Df ---> "Error Term"

Statistic that tells you if the groups are different

(MS b/n) / (MS w/in): "Tx effect + error/error"

• Are grps significantly different from each other?

- If F = 1, Null hypothesis is TRUE (error/error)

-If F > 1, Null hypothesis is FALSE (Tx effect + error/error)

(n(magnitude of effect)/within grp variance)

• Good power when:

1. Large sample size

2. Large effect size

3. Low within grp variance --> Usually want .80 power?

The Magnitude of Effect: To see how big the differences (Tx Effect) are

• Magnitude of effect = Effect Size

• Calculated using R2 or omega2

-Decimal format (example = .60 means that 60% of the variance in the DV is accounted for by the IV)

1. Simple (pair-wise) comparison:

-Comparison b/n 2 grps

2. Complex comparison:

-Comparisons b/n an average of 2+ grps compared to a single grp

3. Orthogonal Comparisons:

-Reflect independent or nonoverlapping pieces of info

-The outcome of one comparison gives no indication about the outcome of another orthogonal comparison

-Coefficient: Weights for the means

 -Y (psi): Difference b/n 2 means

1. Check MAIN EFFECTS: To see if there are differences across levels of one IV

-If no ME, stop analyzing and check power (Power should be around .80)

2. If there is a SIGNIFICANT INTERACTION, look at SIMPLE EFFECTS: How one level of an IV varies across every levle of another

3. Test SE within AxB interaction -

-> Compare As (A @ B1, A @ B2) & Bs (B @ A1, B @ A2)

4. If YES SE: Test for SIMPLE COMPARISONS (only when levels of a variable are >2): A1 v. A2 @ B2 v. A3 @ B1, etc. (Comparisons made within one variable)

ONLY if the Omnibus F is significant!

• BC: Know that the grps are significant, but don't know where the differences are --> Test for differences b/n levels of your IVs

"Family" means grp or set of "related" statistical tests: The probability that a Type I error has been committed in RS involving multiple comparisons

**If alpha =.05 & you make 3 comparisons of the same data, FWE = .15; Could lower Alpha to .01, but this increases probability of TII Error

--> Alternative is to use:

a. Scheffe Test: Adjusts alpha level for all possible comparisons

b. Bonferroni: Strict, Stringent, conservative

c. Sidak-Bonferroni: Not as stringent, more TI, less TII compared with Bonferroni

• Post hoc (after-the-fact) tests: Each mean is compared to each other and you can see where the difference lies

-Important to control for TI Error for each comparison - (# grps -1): Shows you pairwise differences

-The more comparisons you run, the greater the chance for TI Error

--> FWise/ExperWise Error is HIGH

• Type of Correction depends on comparisons:

a. Scheffe:

-Adjusts alpha level for all possible comparisons

-Most stringent

b. Tukey:

-Test differences b/n all possible pairs of means

c. Fisher Hayter: Same as Tukey but uses (a-1)

d. Dunnet:

-Pairwise comparisons using a single group

-eg. Only Tx v. control

e. LSD:

-Least Stringent

• Need to find a good balance b/n making a TI v. TII Error

**Degree of conservativeness of the correction BC of the likelihood of making an error:

Type I Error <----------> Type II Error

Scheffe > Tukey > FisherHayter > Dunnett > LSD

What is the general design of a RM ANOVA?

What are the advantages of a RM ANOVA?

The disadvantages?

The assumptions?

• DESIGN: Doesn't matter how many IVs --> # of IVs in a grp matters (do subjects vary in every condition?

-Time = Most common w/in grp IV

• ADVANTAGES:

1. Can use a smaller sample size BC have more control over subjects' variability

2. Comparing ea person's score against their previous score, not someone else's

• DISADVANTAGES:

1. Practice Effects: Subjects show improvement over time or become bored/fatigued

2. Carryover Effects: Performance on one measure impacts the next (Counterbalance takes care of this problem)

• ASSUMPTIONS:

1. Same as ANOVA

2. Sphericity: Everyone stays in their relative rank (if you were the most anx on the measure, you will be the most anx in every other condition as everyone fluctuates)

-->Greenhouse Geisser tests for this assumption; If violated, look at Huydt Feldt values BC they correct slightly for violations

• Extension of ANOVA that provides a way of statistically controlling the linear effects of variables (covariates, control variables) one does not want to examine in a study

• Reduces experimental error by statistical means

• Subjects 1st measured on the covariate, then randomly assigned to grps without regard for their scores on the covariate

• Covariate should be correlated with the DV, but not with any of the IVs

• Scores on the covariate are used to:

1. Adjust estimates of experimental error

2. Adjust Tx Effect for any differences b/n the Tx grps that existed prior to the experimental Tx

3. Uses Linear Regression to remove covariates from the list of possible explanations of variance on the dependent variable, rather than direct experimental methods to control extraneous variables

4. Used when pretest scores are covariates in pre/posttest exper designs

5. Used in nonexper RS - Surveys, nonrandom samples, quasi-exper designs when RA is not possible

1. Independence of Observation

2. Normality: Examine scatterplot matrices (bi-variate) 3. Homogeneity of Variance: Levene's & Brown Forsythe

4. Linearity: visual examination of scatter plots

5. Homogeneity of Regression: Slope of the regression line (beta) is assumed to be the same for each group, condition, cell

Statistic (Cohen's d, D, delta) indicating the difference b/n grps, Tx, or conditions

• Magnitude of the difference b/n 2+ conditions expressed in standard deviation units: Association of strength or relation (Pearson'r r, eta) 

• To Calculate: ES = (m1-m2)/s

1. Take difference b/n the control and experimental grps' means

2. Divide that difference by the standard deviation of the control grps's scores - or by standard deviations of the scores of both grps combined

• MOE is the size of the tx effect

• Omega squared:

-The proportion of the total variability in the experiment associated with the experimental tx

-Not effected by sample size: Small effect = .01, Large = .15

• R2(square):

- R2 = SSA/SST

-R2 will always be larger than omega: Does not take error into account

• Mz of the relative rand ordering of 2 sets of variables

1. Mean cross-product of all z-scores
2. Tells you the direction and the strength of the relationship
3. Ranges b/n –1 and +1; 0=no relationship
4. When perfect correlation, the scores for all the subjects in the X distribution have the same relative positions as corresponding scores on the Y distribution
5. Correlation will only by high if you have a full range of scores; if not, your correlation will be attenuated (restriction of range)

1. Semi Partial Correlation

2. Partial Correlation

1. Semi-Partial: corr b/n a specific predictor (x) and the criterion (y) when all other predictor in the sfudy have been partialed out of X but not of Y

2. Partial:corr b/n a specific predictor (x) and the criterion (y) when all other predictors have been partialed out of X and Y

Non-parametric alternative to Pearson Product-moment correlation

• Used with ordinal data

• Used to determine the relationship b/n two rank ordered variables (Measured on an ordinal sclae)

-eg. rel b/n knowledge of the political system and self-esteem

- Rank on 2 scales, Spearman's rho measures the association b/n the 2 sets of ranks

-Null hypothesis is that the 2 ranks are independent

The relationship between 2 variables is represented graphically

X variable on (abscissa) horizontal axis and Y on the vertical (ordinate) axis

• Indicates the probability of a true non-zero relationship

-Has nothing to do with the magnitude of the correlation (i.e., just because a correlation is sig at the .001 level does not mean that it is a strong relationship, only that there is a 1% probability that the correlation occurred by chance)

• P value only tells you whether you can interpret the relationship

• Sample size effects p values- NOT the magnitude of the correlation

1. Linearity

2. Bi-variate Normality

3. Full range of scores (no restriction of range)

• Used to determine the utility of a set of predictor variables for predicting an event or behavior (criterion variable)

• MR yields a weighted linear combination of predictors that provides the best prediction of the criterion

• Line of best fit: aka "regression line"

-There is always one straight line that has a smaller sum of squared deviations than any other straight line.

-Regression equation plots a line through the data points that minimizes the residuals (errors)

• Residuals (errors):

- Diff b/n the predicted Y score and the actual Y score

-Mean of the residuals always equals 0

• Y’ = a + bx Y’ = predicted Y

• a = y intercept (the constant)

- Indicates the criterion score when all of the predictors equal 0

-The Y value at which the line touches the vertical Y axis

• b = slope (beta)

-Indicates the effects of the predictor on the criterion

-The expected change in Y for each 1 unit change in X

• x = a score on the x variable

A form of MR that consists of a series of steps in which predictor variables are added (forward inclusion) or subtracted (backward inclusion)

• Change in R2:

-The difference between R2 at each step

-Equals the squared semi-partial correlation coefficient b/n the criterion and the additional predictors that were included in that particular step