What's the difference between Experimental, Observational, and Survey research?

Day 21

• Experimental: directly manipulating IV to look for systematic changes in a DV
• Observational: don't manipulate anything or interact with subjects; simply look at behavior
• Survey: directly questioning participants about behavior and beliefs (no IV manipulation)

What are some drawbacks to surveying (list 7)?

Day 21 

• can't determine causality
  
• answers are at face value (social desirability)
  
• All forms of interview = possible social desirability effects
• non-response bias 
• close-ended questions can limit generality
• participants can misinterpret questions
• priming (wording)                      

What are the 6 steps to creating a survey/questionnaire?

Day 21

1. decide what information to collect
2. decide how to conduct the survey
3. construct a first draft
4. revise
5. pilot
6. edit based on pretest

What are predictor and criterion variables?

Day 21

• Predictor Variables (X) can be used to determine correlations and predictions between participant characteristics and responses (attitude toward abortion can be used to predict voter preference)
• Criterion Variable (Y) is the variable of interest (voter preference)

What are some rules in writing survey items (list 6)?

Day 21

• address one issue per item
• avoid bias/loading
• make alternatives clear
• beware of social desirability
• determine the format of the item
• write mutually exclusive & exhaustive questions

What are the 5 types of questionnaire items?

Give an example of each.

Day 21

• open-ended (free response)
• restricted (multiple choice)
• partially open-ended (multiple choice w/"other")
• rating scale (choose a #)
• likert scales (strongly agree/disagree)

How should you order survey questions (list 4)?

Day 21

1. avoid demographics (stereotype threat)
2. present related items together
3. put more sensitive Qs in the middle or at the end (build rapport/trust)
4. place more objectionable Qs after less objectionable ones

What are the 7 ways to administer a survey?

Day 21

• mail surveys
• internet surveys
• telephone surveys
• group-administered surveys
• face-to-face interviews
• structured interviews
• unstructured interviews

What are two methods in assessing the reliability of a survey?

Day 21

1. Test-retest (alternative or parallel forms)
2. Split-half

What are 3 ways you can increase reliability?

Day 21

1. increase number of items
2. standardized administration
3. clear, simple, non-ambiguous wording

What are the 5 types of validity on a survey?

Day 21

1. Content vailidity (covers range of behaviors)
2. Construct vailidity (correlates with predictions)
3. Criterion-related validity (correlates with related surveys/behaviors, concurrent & predictive vailidity)
4. Convergent validity
5. Divergent validity

What are 3 important components in selecting your survey sample?

Day 21

• Representativeness (sample matches population characteristics random sampling)
• Sampling technique
• Sample size

What are the 9 types of sampling?

Day 21

simple random, stratified, proportionate, systematic, cluster, multistage, convenience, purposive, haphazard

What do each of the variables in the necessary sample size calulation represent?

n' = p(1-p)/(SEp)^2

Day 21

n' = necessary sample size

p = proportion of sample w/target characteristic

SEp = acceptable margin of error

What are the weaknesses of test-retest?

Day 21

• History
• Pre-testing (practice effects)
• Maturation (fatigue & age)
• ideas fluctuate with time
• surveys are long
• retest is administered long after original

What are two ways you can manipulate IVs in experimental research?

Day 22

1. quantitatively (change levels, 10mg-20mg)
2. qualitatively (change category, therapy-drug)

What is the difference between between- & within-subjects designs?

Day 22

• Between: randomly assign different groups of subjects into different levels of IV (different people in each group, independent samples)
• Within: all participants are exposed to every level of IV (same subjects in all groups, non-independent samples)

What are the two main rules of true experiments?

Day 22

1. manipulate the IV directly and completely
2. crontrol for all possible confounds and extraneous variables

What are 4 sources of confounding variables?

Day 22

1. long studies
2. experimenter bias
3. poorly defined or implemented independent variables
4. random sources

What are 6 ways you can reduce error variance?

Day 22

1. hold everything constant
2. randomization & large n
3. effective independent variables (powerful intervention)
4. matched-groups (match subjects in groups; age, gender, etc.)
5. within-subjects designs
6. statistical analysis (inferential stats)

True or False? You can test subjects twice in a between-subjects design.

Day 22

True.

With the exception that you are using a pre & post-treatment

What are the two types of single-factor randomized-groups designs? Describe each.

Day 22

1. randomized two-group design (groups are exposed to only two different levels of the IV)
2. randomized multi-group design (3+ IV levels, ANOVA used, multiple control groups)

In a single-factor randomized multi-group design when would you use a parametric and nonparametric design?

Day 22

• Parametric designs are used when you manipulate your IV quantitatively

• Nonparametric designs are used when you manipulate your IV qualitatively

What does this factorial design mean?

2x2 ANOVA

Day 22

• two levels of two IVs

What's the difference between main and interaction effects?

Day 22

• Main effects are the differences between IVs (row or column means)
• Interaction effects tests to see if your IVs were synergistic (IVs improve treatment) or deleterious (one IV reduces effect of other IV)

What does this factorial design mean?

2x3 ANOVA

Day 22

• Two levels of one IV and three levels of another IV

When would you use a higher-order factorial design?

What is the sample size requirement for using this design?

Day 22

• When you have 3 or more IVs
• need at least 5 subjects/group

What does this factorial design mean and how many subjects are required?

2x2x2 ANOVA

Day 22

• three IVs, two levels of each IV
• minimum of 40 subjects required

What are the steps in selecting and assigning a sample in matched-groups designs?

Day 22

• select pool of subjects
• pretest relevant characteristics
• match subjects into groups based on characteristics
• then randomly assign pairs into groups

What are the two types of matched-groups designs and when do you use them?

Day 22

• matched-pairs design is used when you only have two groups
• matched multi-group design is used when you have more than 2 levels of the IV or when you have more than 1 IV

How many potential main effects are in an 2x2x2 ANOVA?

Day 23

• 3 potential main effects

How many two-way interactions are in a 2x2x2 ANOVA?

Day 23

• 12 two-way interactions

How many three-way interactions are in a 2x2x2 ANOVA?

Day 23

• 8 three-way interactions

What are some disadvantages to using a within-subjects design?

Day 23

• confounds due to fatigue/boredom
• attrition or history effects
• lost data due to measurement/administration errors
• carryover effects

What are 6 sources of carryover effects? Be able to provide an example of each.

Day 23

1. Learning
2. Fatigue
3. Habituation
4. Sensitization
5. Contrast
6. Adaptation

What is counterbalancing? What are the two types?

Day 23

• Counterbalancing is when you assign treatments in a different order for different participants

1. Complete counterbalancing: find every single possible combination of IV orders, present to at least 1 participant
2. Partial counterbalancing:from complete counterbalancing, select only some of the possible combinations, an purposely choose treatment order

When can you not use counterbalancing to approach carryover effects?

Day 23

• When treatments have irreversible effects
• If carryover effects don't have the same approx. magnitude (between conditions)

What are the 3 types of within-subjects designs?

Day 23

1. single-factor two-level design (1 IV, 2 levels. easy to counterbalance)
2. single-factor multi-level design (1 IV, 3+ levels, needs counterbalancing: partial for many IVs)
3. factorial within-subjects design (each participant is exposed to every possible combination of IVs, needs counterbalancing: partial for many IVs)

What are the 5 specialized research designs?

Day 24

1. combined between- and within-subjects
2. combined experimental and correlational
3. quasi-experimental
4. pretest-posttest
5. developmental

When would you use a combined between- and within-subjects design?

Day 24

• when you want to assess the effect of 2+ IVs but irreversible or carry over effects are present

What are the advantages and disadvantages of a combined experimental and correlational design?

Day 24

Advantages: can still test generality, large between-group differences reduces error variance 

Disadvantages: level of quasi-experimental variable may vary, requires additional pretesting

What are the 4 types of Quasi-experimental designs? Be able to destinguish them.

Day 24

1. A-B Time series
2. Interrupted Time series
3. Equivalent Time Sample series
4. Nonequivlent Control Group

When can an equivalent time-series sample design be used?

Day 24

• When behaviors are reversible

What can make a pretest-posttest design a true experimental design?

Day 24

• have a control group and use random assignment

What is a Solomon Four-Group Design?

Day 24

• 50% get pretest, 50% get treatment, 100% get posttest

Which pretest-posttest designs are experimental and which are non-experimental?

Day 25

Experimental:

• Pretest-posttest control group
• Posttest-only control group
• Solomon four-group

Non-Experimental:

• One-group pretest-posttest
  
• One-shot case study
• Static-comparison group
  

What are the three types of developmental designs?

Day 25

• Longitudinal design
• Cross-sectional design
• Cohort-sequential

What are some problems when using longitudinal designs?

Day 25

• multiple-observation effects
• cultural fluctuations
• attrition
• cohort effects
• history effects
• time investment

What are the characteristics of single-subject designs?

Day 26

• repeated measurement
• present all data of all participants
• look at within-subjects changes
• analyzing individual differences
• analyzes both between- & within-conditions
• use visual analysis
• reveals behavior change in relation to repeated exposure

What are some of the advantages of single-subject research?

Day 26

• focus on controlling error variance
• reveals serial patterns of behavior change (trends)
• can find causal relations between IV & DV w/1 subject
• dynamic

What are some disadvantages of single-subject designs?

Day 26

• requires more time & resources
• limited generality (small n)
• carryover effects
• some variance can't be controlled
• not appropriate for some research

Describe what each variable represent in this design:

A-B-A-B-B'

What kind of design is this?

Day 26

A = Baseline

B= IV1

B'= variation of IV1

SSR Single-factor design

What is A-B-A-C an example of? Describe each variable.

Day 26

• This is an example of a SSR multifactor design

A = Baseline

B = IV1

C = IV2

What are the 5 things to look for in a SSR Multifactor design?

Day 26

• Level
• Trend
• Variability
• Stability
• Cycles

What does intrasubject replication mean and what does it establish?

Day 26

• You're able to repeat the effects of IV on DV on the same subject
• each treatment is repeated at least once of each subject
• establishes internal validity

What does intersubject replication mean and what does it establish?

Day 26

• behaviors of multiple subjects are compared
• establishes external validity

In reversal designs, you generally want what 3 things in order to show experimental effect? Describe them.

Day 26

• Demonstration (D), level/trend of DV in treatment1 differs from BL1
• Verification (V), level/trend of DV in baseline2 differs from treatment1
• Replication (R), reintroduction of IV in treatment2 reproduces behavior change observed in treatment1

What are the limitations of an A-B-A design?

Day 26

• ends on baseline (A) instead of treatment
• no indication of experimental control (only D/V are present)

Describe the A-B-A'-B design.

Day 26

• the true "reversal" design
• don't return to baseline, instead you apply IV to a non-target behavior/variable