Term
|
Definition
| Process of reasoning in which the premises of an argument support the conclusion, but do not ensure it. It is used to ascribe properties or relations to types based on limited observations of particular tokens; or to formulate laws based on limited observations of recurring phenomenal patterns. |
|
|
Term
| When do you use inductive reasoning? |
|
Definition
| In specific propositions, such as initial observations or general propositions. |
|
|
Term
| What is an example of an initial observation in inductive reasoning? |
|
Definition
| The ice is cold. The ball moves when you throw it. |
|
|
Term
| What is an example of a general proposition in inductive reasoning? |
|
Definition
| All ice is cold. For each action, there is an equal and opposite reaction. |
|
|
Term
| In inductive reasoning, specific instances lead to... |
|
Definition
|
|
Term
| What is deductive reasoning? |
|
Definition
| Inference in which the conclusion is of no greater generality than the premises, as opposed to abductive and inductive reasoning, where the conclusion is of greater generality than the premises. |
|
|
Term
| What are some examples of valid deductive reasoning? |
|
Definition
The picture is above the desk, the desk is above the floor, therefore, the picture is above the floor.
All birds have wings, a cardinal is a bird, therefore, a cardinal has wings. |
|
|
Term
| In deductive reasoning, general propositions lead to... |
|
Definition
|
|
Term
| What are some examples of invalid deductive reasoning? |
|
Definition
Lemons are a citrus fruit, my car is a lemon, therefore, my car is a citrus fruit.
All baby goats are kids, I have a kid, therefore, I have a baby goat. |
|
|
Term
| With regard to reasoning, a question you may want to ask yourself for grants/theses is, "Are your hypothesis... |
|
Definition
| Dependent on one another?" |
|
|
Term
| What is infinite regress? |
|
Definition
| In a series of propositions, arises if the truth of the proposition in P1 requires the support of proposition P2, and for any proposition in the series Pn, the truth of Pn requires the support for Pn+1, because the infinite series needed to provide such support could not be completed. |
|
|
Term
| What is a tautological argument? |
|
Definition
| Otherwise known as a circular argument, that is, one that begins by assuming the very thing that is meant to be proven by the argument itself. Tautological arguments are not really arguments at all; they assume facts yet not in evidence. |
|
|
Term
|
Definition
| The use of redundant language in speech or writing (e.g., saying the same thing twice). |
|
|
Term
| What are some examples of tautologies? |
|
Definition
| Non-cognate synonyms (e.g., helpful assistance, a three-part trilogy) or repetition of an abbreviated word (e.g., ATM machine, PIN number). |
|
|
Term
| Science proceeding from the bottom up is _____ --> _____, whereas science that proceeds from the top down is _____ --> _____. |
|
Definition
| Observations --> theories; Theories --> observations. |
|
|
Term
| Popper argues that science proceeds through _____ methods. |
|
Definition
|
|
Term
| What are Hume's four arguments about science? |
|
Definition
| (1) Beliefs through general laws are attained by inductive inference, (2) Inductive inference is unjustified because inferences can be disproven, (3) If a belief is not justified, then it does not count as knowledge, and (4) As a result, we cannot have knowledge about general laws. |
|
|
Term
| What are Popper's arguments in response to Hume? |
|
Definition
| (1) Denies that scientists use inductive methods at all, (2) Theories are conjectural and development of theories is not necessarily a logical matter, (3) The testing of a theory, on the other hand, can proceed along logical lines, and (4) Illogical theories can be systematically tested? |
|
|
Term
| According to Popper, along which four lines does the evaluation of a theory proceed? |
|
Definition
| (1) Logical comparison of conclusions yielded by the theory (evaluation of internal consistency), (2) Investigation of the logical form of the theory (empirical? truly scientific? tautology?, logical forms can sometimes be circular), (3) Comparison with other theories (is it an advance? paradigm shifts? theory cannot explain everything), and (4) testing theory through empirical means (does new theory explain better than old and explain issues the old could not?) |
|
|
Term
| Thomas Kuhn: Science is like... |
|
Definition
| Politics because (a) your theory cannot explain everything despite infinite alterations, and (b) paradigm shifts. |
|
|
Term
| Experiments will yield statements that either _____ or _____ the theory. |
|
Definition
| Support (verify); Do not support (falsify). |
|
|
Term
| If a theory withstands a host of empirical tests, it should not be... |
|
Definition
|
|
Term
| Two good reasons for discarding a theory? |
|
Definition
| (1) Replacement of a theory or attendant hypothesis by another hypothesis that can account for results better, or (2) Falsification of one of the consequences of the hypothesis. |
|
|
Term
| True or false: Corroboration and verification are the same thing as saying the theory is true in any meaningful sense. |
|
Definition
| False, corroboration is testing the individual elements of a theory; no theory can ever be completely "true," only supported. |
|
|
Term
| True or false: There could always be better theories to account for facts. |
|
Definition
|
|
Term
| True or false: No other variables can alter our conclusions besides what we are already taking into account. |
|
Definition
| False, other variables may alter our conclusions and we may not always be taking these into account. |
|
|
Term
| Popper: No conclusive _____ of a theory can ever be produced. |
|
Definition
|
|
Term
| In the example given during lecture, Galton developed a theory of inherited intelligence and success. Based on the science of theories, what did he do wrong? |
|
Definition
| Galton forgot to take into account monetary advantages. He did not consider all the variables that may alter conclusions. |
|
|
Term
| If the aim of science is to provide true theories that have been verified, what is the best way of doing this? |
|
Definition
| Generalizations cannot be conclusively verified, but can be conclusively falsified. |
|
|
Term
| Trying to "confirm" theories by observing more and more positive inferences does... |
|
Definition
| Nothing toward proving a theory. Theories can never be proven, just supported or falsified. |
|
|
Term
| What is the cornerstone of Popper's approach? |
|
Definition
| Falsification, unlike verification, can weed out false theories, which is done by trying our best to falsify or refute them. |
|
|
Term
| Popper's perspective is the opposite of... |
|
Definition
| An inductivist perspective. |
|
|
Term
| What does an inductivist believe? |
|
Definition
| Allow into body of knowledge only those theories which one has good reason to believe to be true. |
|
|
Term
| What does a falsificationist believe? |
|
Definition
| The way to truth is to allow any theory into one's body of knowledge and then expel the false ones. |
|
|
Term
| What distinguishes science from non-science? |
|
Definition
| Falsifiability. Statements about theories are falsifiable, not the theories themselves. |
|
|
Term
| Popper's claim regarding falsifiability: |
|
Definition
| "A theory is scientific if, and only if, it is falsifiable by empirical evidence." |
|
|
Term
| Determining falsifiability always involves methods of... |
|
Definition
|
|
Term
| True or false: You cannot infer from a statement of theory whether it is a scientific statement. |
|
Definition
|
|
Term
| How are falsifiability and scientific vigor related? |
|
Definition
| They have a positive relationship. |
|
|
Term
| Theoretical "patches" (e.g., alterations to your theory) decrease falsifiability. What does this mean for the quality of your theory? |
|
Definition
| It decreases the quality of your theory. |
|
|
Term
| True or false: The proponents of a method do not have a way of falsifying it, making this a scientific theory. |
|
Definition
| False, if the proponents of a method CAN falsify it, then it is a scientific theory. |
|
|
Term
| According to Popper, Marxism, Adlerian psychology, and Freudian psychoanalysis are _____ because... |
|
Definition
| Unscientific; proponents of the theories refuse to count any possible observation as refutation. |
|
|
Term
| True or false: An unfalsifiable theory might still be true, it's just not scientific. |
|
Definition
|
|
Term
| The basic aim of science is |
|
Definition
|
|
Term
| What are theories built upon? |
|
Definition
| Description, classification, observations of functional relations/co-variations, and speculations of causality. |
|
|
Term
| According to Popper, it is impossible to evaluate the truthfulness of a theory, so what do we do instead? |
|
Definition
| We evaluate individual statements derived from the theory/hypothesis provided those statements are expressed in such a way that they can be falsified. |
|
|
Term
|
Definition
| A set of interrelated constructs, definitions, and propositions that present a systematic view of phenomena by specifying relations among variables, with the purpose of explaining and predicting phenomena. |
|
|
Term
| Do we truly understand phenomena? |
|
Definition
| Our degree of understanding is never truly known. |
|
|
Term
| What is the chief goal of the scientist? |
|
Definition
|
|
Term
| Two primary ways of evaluating theories: |
|
Definition
| (1) Falsifiability of statements (good theories cannot fit all possible explanations and certain events should disprove propositional statements consistent with the theory), and (2) Parsimony (when both a simple and complex theory account for the facts equally well, the simplest explanation is preferred). |
|
|
Term
|
Definition
| The principle of parsimony, which states that when both a simple and complex theory account for the facts equally well, the simplest explanation is preferred. |
|
|
Term
| How do we test propositional statements? |
|
Definition
| Propositional statements imply a relationship between variables, so we test the relationship between variables, not the variables themselves. We test the functional relation that is hypothesized, which sometimes may be assumed to be a causal relation. |
|
|
Term
| Propositional statements are... |
|
Definition
| Implications from hypotheses (not direct hypotheses) or experimental hypotheses test proxies. They are typically deduced from broader hypotheses. |
|
|
Term
| If your hypothesis is not supported and your theory is falsified, what could be the problem? |
|
Definition
| Issues in methods or you need to make ad hoc changes to your model (especially good to make these changes if they increase falsifiability). |
|
|
Term
| A good problem statement should... |
|
Definition
| Express a relation between two or more variables (under certain conditions, with certain variables, etc.), be stated clearly/unambiguously in question form, and imply possibility of empirical testing (e.g., you should be able to deduce the analyses to be run). |
|
|
Term
| Hypothesis testing implies the notion that... |
|
Definition
| The test can be failed, which is crucial, otherwise is it not really a test. |
|
|
Term
| What were the primary methods for data analysis in the 19th century? |
|
Definition
| Graphing data and implying differences between groups in that way. |
|
|
Term
| Galton developed the notion of... |
|
Definition
|
|
Term
|
Definition
|
|
Term
| How did Galton utilize covariations? |
|
Definition
| By repeatedly graphing things. This was similar to multiple regression inferential statistics in that graphing also found interaction effects. |
|
|
Term
| Why do many researchers question the utility of inferential statistics? |
|
Definition
| Because many long-lasting contributions to psychology were made by those who did not use inferential statistics (e.g., Freud, Skinner, Piaget). |
|
|
Term
|
Definition
| An insufficient but non-redundant part of an unnecessary but sufficient condition. We wanted to see the effects of interventions, which is why we created inferential statistics. |
|
|
Term
|
Definition
| Knowledge of what would have happened but for the occurrence of something (hypothetically speaking, your intervention can be the cause or counterfactual). |
|
|
Term
|
Definition
| The difference between what did happen and what would have happened. |
|
|
Term
| In causal relationships, which requirements do you need to say that A caused B? |
|
Definition
| Cause preceded effect, cause was related to effect, and there is no plausible alternative explanation. |
|
|
Term
|
Definition
| A systematic covariation of two independent variables. |
|
|
Term
| How is causality assessed? |
|
Definition
| Begin by holding all other causal variables constant. You assess the level of a variable, implement experimental manipulation, and assess level of variable post-manipulation (basically, the pretest-posttest design). |
|
|
Term
| What are two comparisons that can be made when assessing a level of a trait pre- and post-manipulation of the experimental group? |
|
Definition
| (1) Compare across groups (e.g., same at pretest, different at posttest) and (2) Compare across time (e.g., changes in experimental group and not in control group). |
|
|
Term
| What is the cost of pre- and post-manipulation of experimental groups and leaving everything else constant? |
|
Definition
| Generalizability, because in reality, patients choose the type of treatment they would like to receive. |
|
|
Term
| Why is generalizability compromised in some experiments? |
|
Definition
| To achieve enhanced internal validity. |
|
|
Term
| What are the two steps to generalizability? |
|
Definition
| Random sampling and random group assignment. |
|
|
Term
| What is the two-step process of random sampling? |
|
Definition
| Random selection and random assignment. |
|
|
Term
| What are three types of random sampling? |
|
Definition
| Simple random sampling, cluster sampling, and stratified random sampling. |
|
|
Term
| What is simple random sampling? |
|
Definition
| Assume random selection but rarely happens because it is impossible to enumerate everyone in the population. This type of sampling assumes all members are accounted for and they all have an equal probability of being selected. |
|
|
Term
| What is cluster sampling? |
|
Definition
| Dividing the population into clusters and randomly and proportionately drawing from the clusters. It requires that you know something about your population. |
|
|
Term
| What is stratified random sampling? |
|
Definition
| Dividing the universe into relevant strata based on demographic variables. It requires that you know something about your population. |
|
|
Term
| Clustering _____ estimates of the population parameters. |
|
Definition
|
|
Term
| Stratifying _____ estimates of the population parameters. |
|
Definition
|
|
Term
| True or false: Random sampling frequently occurs. |
|
Definition
| False: Random sampling rarely occurs. |
|
|
Term
| According to the textbook, the two-step model of random sampling followed by random assignment cannot be advocated as the model of... |
|
Definition
| Generalized causal inference. |
|
|
Term
| The best way to design a study is to... |
|
Definition
| Randomize the selection and assignment. |
|
|
Term
| What is the alternative to the goal of producing findings that generalize to different people, settings, treatments, or measurement variables? |
|
Definition
| Understanding why findings do not generalize to different people, settings, treatments, or measurements variables caste within a general theory. |
|
|
Term
| True or false: You are more interested in your population than your sample. |
|
Definition
|
|
Term
|
Definition
| Approximations of truth regarding causal inferences. In the generalizability context, it is that your beliefs that A causes B are valid. |
|
|
Term
| Define construct validity. |
|
Definition
| The degree to which inferences can legitimately be made from the operationalizations in your study to the theoretical constructs on which whose operationalizations were based. |
|
|
Term
| According the theory of generalizability, what are the components of a theory? |
|
Definition
| Cause construct --> Cause-effect construct --> Effect construct. |
|
|
Term
| According to the theory of generalizability, what are the components of an observation? |
|
Definition
| Program/intervention --> Intervention-outcome relationship --> Observations/test scores |
|
|
Term
| According to the theory of generalizability, how do the components of a theory relate to the components of an observation? |
|
Definition
| Cause construct is related to program/intervention, cause-effect construct is related to intervention-outcome relationship, and effect construct is related to observations/test scores. |
|
|
Term
| Define external validity. |
|
Definition
| The extent to which our findings generalize to other persons, settings, treatments, or outcomes. |
|
|
Term
| What are the five principles of generalized causal inference? |
|
Definition
| (1) Surface similarity, (2) Ruling out irrelevancies, (3) Making discriminations, (4) Interpolating and extrapolating, and (5) Causal explanation. |
|
|
Term
| What is an example of surface similarity? |
|
Definition
| Since similar is knowing if findings generalize, an example would be Seligman generalizing his research findings of learned helplessness in rats to humans. |
|
|
Term
| What is an example of ruling out irrelevancies? |
|
Definition
| Determining that the height of felames does not matter for treatment success of depression. |
|
|
Term
| What is an example of making discriminations? |
|
Definition
| Since it is the opposite of ruling out irrelevancies, an example of making irrelevancies would be saying that treatment worked for women of varying heights, but will be ineffective for men. |
|
|
Term
|
Definition
| Extending hypotheses in the range of the measure. |
|
|
Term
|
Definition
| Extending beyond assessed range. |
|
|
Term
| What is a causal explanation? |
|
Definition
| A causal explanation involves mechanisms of action and explanatory theories that will hold under a variety of conditions. |
|
|
Term
| According to the textbook, the five principles of generalized causal inference have implications for which two types of validity? |
|
Definition
|
|
Term
| What are some threats to construct validity? |
|
Definition
| Person, setting, treatment, outcome, different types of constructs, being a scholar in isolation (versus an independent scholar). |
|
|
Term
| What is an inadequate explication of a construct? |
|
Definition
| A threat to construct validity that occurs when there is a mismatch between what you want to measure and the operations used to measure it. |
|
|
Term
What is construct confounding?
|
|
Definition
| A threat to construct validity that occurs because our operations are rarely a pure representation of what we are interested in. |
|
|
Term
| What is a mono-operation bias? |
|
Definition
| A threat to construct validity that occurs when facets that are over- or underrepresented are measured in multiple ways. This can include several measures of a given construct. If these findings are not congruent, you may present both findings or exclude one of them. |
|
|
Term
| What is a mono-method bias? |
|
Definition
| A threat to construct validity that occurs when presentation (e.g., administration of treatment) produces an effect. |
|
|
Term
| How is confounding constructs with levels of constructs a threat to construct validity? |
|
Definition
| Because results differ with the level of the construct studied. |
|
|
Term
| What is a treatment-sensitive factorial structure? |
|
Definition
| A threat to construct validity that is a way of conceptualizing treatment effects. Treatment affects the presence of a construct and factor structure. This type of structure is like a second wave CFA in that there is a different factor structure across groups (e.g., changes in the level of the construct presented). |
|
|
Term
| What are some other more specific threats to construct validity? |
|
Definition
| Inadequate explication of a construct, construct confounding, mono-operation bias, mono-method bias, confounding constructs with levels of constructs, treatment sensitive factorial structure, reactivity to self-report changes, reactivity to experimental changes, and experimenter effects, mismatching cause construct and program/intervention, mismatching effect construct and observations, reactivity confounds (novelty and disruption effects, compensatory equalization, compensatory rivalry, resentful demoralization, treatment diffusion), interactions of the causal relationship (with units, over treatment variations, with outcomes, and with settings), context dependent mediation, constancy of effect size, constancy of causal direction, and purposive sampling. |
|
|
Term
| What occurs in an interaction of the causal relationship with units? |
|
Definition
| Effects found for some units are not found in other units. |
|
|
Term
| What is an example of an interaction of the causal relationship over treatment variations? |
|
Definition
| One medication alone versus one medication in combination with other medications have different effects on individuals. |
|
|
Term
| What is an interaction of the causal relationship with outcomes? |
|
Definition
| Cause-and-effect may not generalize across outcomes (e.g., are treatments equally effective in treating different facets of constructs?). |
|
|
Term
| What is an interaction of the causal relationship with settings? |
|
Definition
| Some effects occur in some settings and not in others. |
|
|
Term
| Why is context-dependent mediation important? |
|
Definition
| Identification is a necessary process in order to transfer an effect. |
|
|
Term
| What are some things to consider when you want to sample to get diversity? |
|
Definition
| Persons, settings, treatments, and outcomes. |
|
|
Term
| Define statistical conclusion validity. |
|
Definition
| Inference made about the covariation between treatment and outcome. |
|
|
Term
| Define internal validity. |
|
Definition
| Impact of treatment on outcome. |
|
|
Term
| What are threats to statistical conclusion validity? |
|
Definition
| Low statistical power, violated assumptions of statistical tests, alpha inflation due to data fishing, unreliability of measures, restriction of range, unreliability of treatment implementation, extraneous variance in experimental setting, heterogeneity of units, and inaccurate effect size estimation. |
|
|
Term
| What are some ways to increase power? |
|
Definition
| Using matching/stratifying/blocking, measure and correct for covariates, use larger samples, use equal cell sample sizes, improve measurement, increase strength of treatment or the extent to which groups differ, increase variability of treatment. |
|
|
Term
| Poor measurement leads to poor |
|
Definition
|
|
Term
| How do you increase the strength of a treatment? |
|
Definition
| Tightened construct measures with low standard error. If you tighten these measures, you will see smaller standard deviations via the narrower normal distribution. |
|
|
Term
| What is a common reason for why people violate the assumptions of their statistical test? |
|
Definition
| They forget what they are. |
|
|
Term
| How do you correct for alpha inflation due to data fishing? |
|
Definition
| Use an adjustment statistic (e.g., Bonferroni). |
|
|
Term
| How does unreliability of a measure threaten statistical conclusion validity and what is a way to prevent this? |
|
Definition
| It attenuates the bivariate correlation between variables. To prevent this, disentangle variances and look at different levels of the construct (can be done through SEM). |
|
|
Term
| How does restriction of range threaten statistical conclusion validity? |
|
Definition
| It reduces the power and attenuates the bivariate correlation between variables. |
|
|
Term
| What types of studies are susceptible to unreliability of treatment implementation? |
|
Definition
| Large studies or studies with administration at different sites. |
|
|
Term
| How does extraneous findings in the experimental setting threaten statistical conclusion validity? |
|
Definition
| They produce covariate findings that are inaccurate due to the setting (distractions, noise, change of location, etc.). |
|
|
Term
| Besides threatening statistical conclusion validity, what happens if units are too heterogeneous in the dependent variable? How do you control for this? |
|
Definition
Standard deviation increases and significance of the treatment effect decreases.
Control for this by creating a statistical control for relevant covariates or using confounds as blocking variables. |
|
|
Term
| What is the implication for inaccurate effect size estimation and what is an example in which this could occur? |
|
Definition
| Inaccurate effect size estimation threatens statistical conclusion validity and decreases effect size dramatically. An example of this could be having many outliers that deviate from normality. |
|
|
Term
| What is an alternative to null hypothesis testing? |
|
Definition
| Computing confidence intervals. This helps to distinguish between situations of low statistical power, and hence wider confidence intervals, and situations with precise, but small effect sizes. |
|
|
Term
| What are threats to internal validity? |
|
Definition
| Ambiguous temporal sequence, selection, history, maturation, regression (to the mean), controls, attrition, testing, instrumentation, and additive/interactive effects of threats to internal validity. |
|
|
Term
| Internal validity is the sin qua non of... |
|
Definition
| Experimental science (changes are a direct consequence of intervention). |
|
|
Term
| What is ambiguous temporal sequence? How do you prevent this? |
|
Definition
| Order of the variables in the causal relationship is unknown. Prevent this by using experimental designs. |
|
|
Term
| How does selection threaten internal validity? What is a way to prevent this threat? |
|
Definition
| People on different conditions differ at the start of the experiment. Random assignment may increase comparability, but this isn't guaranteed. |
|
|
Term
| How does history threaten internal validity and how is this threat prevented? |
|
Definition
| Anything occurring between the beginning of treatment (pretest) and posttest could have produced a desired outcome. Prevent this by using the Solomon-4 group design (a quasi-experimental design), which increases duration and the probability that this will happen. |
|
|
Term
| How does maturation threaten internal validity? How do you control for this? How do you prevent it? |
|
Definition
| People change over time and this change can be assumed to be due to intervention when it happened outside of intervention. It can be controlled for by sampling different geographical regions or using a cross-sequential design. It can be prevented altogether by using the Solomon-4 group design. |
|
|
Term
| What occurs in a regression to the mean and how can it be prevented from threatening internal validity? |
|
Definition
| A regression to the mean occurs because when extreme-scoring units are selected, they will often have fewer extreme scores on other variables, which could be misconstrued as a treatment effect. This is prevented by obtaining a large set of extreme scorers and randomly assigning them to different treatments to regression will occur equally across groups. |
|
|
Term
| What are some ways to increase reliability of assessment? |
|
Definition
| Use longer more reliable instruments, use multivariate function, and averaging scores over two assessment points. |
|
|
Term
| What is the three time method and what does it allow for? |
|
Definition
| Selection based on time 1, implementation of treatment in time 2, and final assessment in time 3. It allows for comparisons of T2 and T3 versus T1 and T3. |
|
|
Term
| What is attrition, how can it be prevented from threatening internal validity, and how is it problematic? |
|
Definition
| Attrition is the loss of units after randomization has taken place which can produce effects if the loss is systematically correlated with the conditions of the experiment. It can be prevented by using a pretest-posttest control group design. It is problematic when those who remain differ from those who drop out and if it is due to the treatment (e.g., side effects). |
|
|
Term
| How can testing threaten internal validity and how can this be prevented? |
|
Definition
| Exposure to a test/items can affect scores on the exposed items (practice effects), producing an effect mirroring intervention. To prevent this, use IRT or Solomon-4 group design. |
|
|
Term
| How does instrumentation threaten internal validity and how can this be prevented? |
|
Definition
| Changes in an instrument over time can mimic treatment effects. Prevent this by calibrating instruments, especially if you switch instruments. |
|
|
Term
| What are the additive/interactive effects of threats to internal validity? |
|
Definition
| The impact of one threat can add to the impact of another threat, depending on severity. |
|
|
Term
| If you accept the null hypothesis and it is true... |
|
Definition
| You correctly failed to reject the null hypothesis. |
|
|
Term
| If you reject the null hypothesis and it is false... |
|
Definition
| You correctly rejected the null hypothesis, increasing power. |
|
|
Term
| If you accept the null hypothesis and it is false... |
|
Definition
| You incorrectly fail to reject the null hypothesis, leading to Type II error. |
|
|
Term
| If you reject the null hypothesis and it is true... |
|
Definition
| You incorrectly reject the null hypothesis, leading to Type I error. |
|
|
Term
| What is Type I error and what is the probability of it? |
|
Definition
| Type I error is concluding that the null hypothesis is false (or should be rejected) when it is actually true (and should not be rejected). The probability of Type I error is alpha. |
|
|
Term
| What is Type II error and how is probability of it determined? |
|
Definition
| Type II error is concluding that the null hypothesis is true (and should not be rejected) when it is actually false (and should be rejected). This is represented by beta and is the complement of power. The probability of Type II error is determined by many factors, one of which is its reciprocal relationship with Type I error. |
|
|
Term
| How are Type I error and Type II error related? |
|
Definition
| They have a reciprocal relationship. |
|
|
Term
|
Definition
| The raw magnitude of an effect. In t-tests, it is the observed mean difference between groups on a measure. In the simplest of terms, it is the difference between groups. |
|
|
Term
| _____ effects are more impressive than _____ effects. |
|
Definition
|
|
Term
| What are advantages of using a raw effect size? |
|
Definition
| Expected value is independent of sample size and it is expressed directly in terms of the units of scale of the dependent variable. |
|
|
Term
| What are disadvantages of using raw effect size? |
|
Definition
| With small-ish samples, you may be able to obtain an effect without being able to reject the null hypothesis, dangerous to judge effect size in isolation from p-value, investigator must be comfortable/experienced with scale of measurement. |
|
|
Term
| What is standardized effect size? |
|
Definition
| Effect size in standard deviation units (scale-independent), it is the raw effect size divided by the standard deviation of scores (within groups) on the response scale. |
|
|
Term
| What are advantages of using standardized effect size? |
|
Definition
| Independence of response scale if combining studies with different response scales (e.g., meta-analyses) - useful if you want to compare effectiveness of various/different measures/interventions and can take advantages of percentage calculations based on normal distributions. |
|
|
Term
| What are disadvantages of using standardized effect size? |
|
Definition
| Cannot tell us whether the null hypothesis can be rejected and depends on sample size. |
|
|
Term
| What is a way in which effect size can be better than a p-value? |
|
Definition
| Effect size gives us some better feel for the magnitude of treatment effects; p-value does not. |
|
|
Term
| Sometimes, small effect sizes are impressive when |
|
Definition
| Not much effort went into the manipulation. |
|
|
Term
| The magnitude of the effect can be modified by |
|
Definition
|
|
Term
| We find results interesting when the manipulation of the independent variable is minor and |
|
Definition
|
|
Term
| What is subjective causal efficacy? |
|
Definition
| Effect size based on topic (how important it is). |
|
|
Term
| Methodological rigor and importance can increase |
|
Definition
|
|
Term
|
Definition
| The probability that a significance test will lead to the rejection of the null hypothesis when the null hypothesis is indeed false. It is the complement of Type II error (one minus beta). |
|
|
Term
| What are ways of increasing power? |
|
Definition
| Significance level adopted, effect size, reliability of sample data (standard error of mean differences), homogeneous groups (you want tx and control to be as different as possible!), "approaching" significance (BAD), unconventional cut-offs (also BAD). |
|
|
Term
| What is the formula for the standard error of mean differences? |
|
Definition
| Sxbar-ybar = √(1/n1 + 1/n2)σ2, where Sxbar-ybar is the standard error of the difference between two means and σ2 is the pooled estimate of the (assumed equal) population variances. |
|
|
Term
| Standard error of the mean difference has an inverse relationship with |
|
Definition
|
|
Term
| Standard error of the mean is similar to variability in that as your variability increases, your distance to the critical value (mean) ______ in order for you to have a significant effect. |
|
Definition
|
|
Term
| What does randomization do? |
|
Definition
| (1) Reduces plausibility of alternative explanations for treatment effects, (2) Yields unbiased estimates of the average treatment effects, (3) Allows counterfactual inferences, (4) Ensures cause precedes effect, and (5) Tests for significant differences between groups. |
|
|
Term
| What are limitations of randomization? |
|
Definition
| (1) The only internal validity threat it controls is selection bias and (2) It doesn't prevent or control things like regression to the mean, maturation, etc. |
|
|
Term
| How does randomization work? |
|
Definition
| (1) Increases probability that alternative causes are not confounded with the unit's treatment condition, (2) Reduces the plausibility of threats to validity by "disturbing" them randomly across conditions, (3) Allows the researcher to know and model the selection process, (4) Allows computation of a valid estimates of the error of variance and is orthogonal to treatment, and (5) Equates groups on the expected value of all variables present at pretest (whether they're measured or not). |
|
|
Term
| You want groups to be the same at pretest because |
|
Definition
| You will equally distribute potential confounds between groups. |
|
|
Term
| What is the formula explaining how randomization works? Explain what this formula actually means. |
|
Definition
| Yi = U + βZi + ei, with Yi being the dependent variable, U being the constant, β being the regression coefficient, Z being the independent variable, and e containing the potential confounds. Since randomization tries to guarantee the probability that the correlation between beta and erorr is zero, this assumes that the dependent variable is equal to the constant at pretest. This formula demonstrates exposure versus placebo. Randomization estimates error variance (variability within group not attributable to treatment = error). |
|
|
Term
| What are types of controls |
|
Definition
| No treatment, dose-response, placebo, wait-list, expectancy, and deconstructed elements of total treatment. |
|
|
Term
| What is dose-response control? |
|
Definition
| Controls for the magnitude or salience of treatment (drug amount, number of sessions, etc.). |
|
|
Term
|
Definition
| Controls for inert aspects of intervention. |
|
|
Term
| What is wait-list control? |
|
Definition
| Controls for time passage. |
|
|
Term
| What is expectancy control? |
|
Definition
| Systematically manipulates the beliefs about treatment. |
|
|
Term
| What are the deconstructed elements of total treatment? |
|
Definition
| When the control is a portion of the treatment (e.g., dismantling studies), it is essential to consider what you're trying to control for (covariates, the causal agent in the intervention). |
|
|
Term
| Basic randomized design comparing two treatments is the alternative to ______ and it is used when... |
|
Definition
| Basic design; developing a new treatment you believe is more effective than an existing "gold standard" treatment by comparing the treatments (one of which must be established). |
|
|
Term
| What does no pretreatment assessment do and what are advantages/disadvantages of this design? |
|
Definition
| No pretreatment assessment compares those who drop out from different conditions and compares those who do and do not drop out. It is used when both treatments are not established and you need a control. Advantages to this is that it circumvents problem of sensitization and the pretest would mirror intervention. A disadvantage to this is that it can't assess reasons for attrition. |
|
|
Term
| What are some characteristics of pretest/posttest control group design? |
|
Definition
| Both types of this design differ in whether testing occurs before or after assignment. This type of design copes with attrition as a threat to internal validity and has statistical advantages. It is the most commonly used randomized field design. It allows you to analyze/reject the null hypothesis. You want to make the pretest as similar as possible to the posttest (IRT is alternative to this). Matched groups design. |
|
|
Term
| For alternative-treatment with pretest and multiple treatment with controls and pretest, if no group differences exist... |
|
Definition
| Examine the pretest and the posttest to see if both groups got better or if there was no change. If there was a slight decline, it is likely that there was a regression of the mean. |
|
|
Term
| What does the Solomon-4 group design test for and what does it control for? |
|
Definition
| It is a very important and advantageous design that tests for the effect of maturation and history. It controls for sensitizing effects by including groups just assessed at posttest. |
|
|
Term
| What are some characteristics of factorial designs? |
|
Definition
| Composed of two or more independent variables each with at least two levels, require fewer subjects, test treatment combinations easily, allow for testing of interactions or moderators, and filling all cells increases power. |
|
|
Term
| What is a fractional factorial design? |
|
Definition
| Not all the cells are filled because a combination doesn't make sense or is not important. Other than that, it is very similar to a regular factorial design. |
|
|
Term
| What are some issues in factorial design? |
|
Definition
| Sample sizes and potential problems with needing to recruit large sample sizes and nesting versus crossed designs. |
|
|
Term
|
Definition
| Some levels of one factor are not exposed to all levels of another factor. This may not yield unconfounded statistical tests of all main effects and interactions. In fact, it usually won't. |
|
|
Term
| What is a crossed design? |
|
Definition
| Each level of a factor is exposed to all levels of other factors. It yields unconfounded statistical tests of all main effects and interactions. |
|
|
Term
|
Definition
|
|
Term
|
Definition
|
|
Term
| What is partial eta squared? |
|
Definition
| Effect size, the proportion of total variance attributable to the factor, partialling out other factors. |
|
|
Term
| Treatment implementation involves |
|
Definition
| Delivery (fidelity issues, sending the message you need to send), receipt (message understood), and adherence (desired behavior). |
|
|
Term
| What are ways to enhance treatment delivery (treatment fidelity)? |
|
Definition
| Use of treatment manuals (universal), (directly) train service providers (researchers), provide continuing training experiences (like supervision), and video/audiotaping and reviewing. |
|
|
Term
| What are ways to measure treatment delivery (treatment fidelity)? |
|
Definition
| Scoring or reviewing the video/audiotapes (independent raters of fidelity in tapes) and discuss progress at informal staff or research meetings (meet regularly). |
|
|
Term
| What are ways to enhance treatment receipt? |
|
Definition
| Give written handouts to participants (delivery of message in multiple ways), summarize key treatment or study elements, use repetition of message (make sure message is made clear), ask participant questions to improve information encoding (make sure delivery is credible), and use a researcher/delivery person who is attractive or seems like an expert (enhance credibility). |
|
|
Term
| What are ways to measure treatment receipt? |
|
Definition
| Use manipulation checks, give written tests, and monitor changes (cognitive, physiological, attitudinal, etc.) that should be observed if the treatment is effective. |
|
|
Term
| What are ways to enhance treatment adherence? |
|
Definition
| Make it personal (anytime you want to see someone on multiple occasions), reduce amount of time needed (make it simple), incorporate frequent reminders, make certain to recognize successes early in treatment, create homework assignments that can be done in a variety of locations, use family members and friends to encourage participant, use tape recordings or other AV aids, and raffle tickets/incentives. |
|
|
Term
| What are ways to measure treatment adherence? |
|
Definition
| Regularly monitor each enhancement, and use bio assays if possible (classically used in smoking cessation). |
|
|
Term
| What is intent to treat analysis? |
|
Definition
| A type of analysis based on the amount of treatment received and if you plan on treating all participants as if they received the full package of the experiment. It assumes unbiased estimators if the dropouts are random. With dropouts, it takes the last observed value during participation and carries it forward through the rest of the experiment. This is a quasi-random design because it preserves the advantages of random assignment. |
|
|
Term
| What is post-assignment attrition and how can it occur? |
|
Definition
| Any loss of response from participants that occurs after participants are randomly assigned to conditions. It may happen due to a failure to answer a single question or completely dropping out. The participant or research may initiate the drop, but it is rarely a good idea to deliberately drop participants after assignment. |
|
|
Term
| Moderate to high attrition is common particularly when... |
|
Definition
| Treatment is aversive, disorder is particularly prone to high rates of relapse (e.g., substance abuse), treatment demands are high (a lot of work on participants), and the assessment demands are high or the population is highly mobile. |
|
|
Term
| Attrition is less often "random" and more often biased because... |
|
Definition
| People may drop out of one treatment more than another or those who drop out may differ from those who remain. |
|
|
Term
| The burden of proof is on the researcher to prove that attrition is which two things? |
|
Definition
| (1) Not treatment-related, and (2) Did not influence the apparent outcome. |
|
|
Term
| True or false: Sometimes, dropping a subject is inevitable. |
|
Definition
|
|
Term
| Retention and tracking strategies to prevent post-assignment attrition involve collecting demographic information of... |
|
Definition
| Participants, relatives, collaterals, and professionals in the community with whom they have contact. |
|
|
Term
| Front-end demographics are used for... |
|
Definition
| Retention or comparing sample with dropouts. |
|
|
Term
| What is the real problem with attrition? |
|
Definition
| How to handle dropouts and/or missing data. |
|
|
Term
| What is a common strategy for replacing dropouts and how is this strategy effective? |
|
Definition
| Replacing dropouts with randomly selected persons drawn from the same pool, which retains power/sample size. This strategy is only effective if attrition and replacement are random (unlikely), and both the former and replacements have the same latent characteristics (measured and unmeasured). This is unfortunately not a very good strategy. |
|
|
Term
| What are some things simple descriptive statistics can tell us about attrition? |
|
Definition
| Overall attrition rate, differential attrition rates for groups, whether completers and non-completers differ on important characteristics, whether those who completed the treatment differed from those who did not, and whether those who completed placebo differed from those who did not. |
|
|
Term
| Prior to analyzing the data, what are some ways to look for patterns of attrition? |
|
Definition
| Whether different groups have different patterns of attrition, whether different measures have different attrition patterns, and whether subsets of respondents or sites have complete data that could be used to salvage some randomized comparisons. |
|
|
Term
| How can you account for attrition when estimating effects? |
|
Definition
| Impute values for missing data, bracket possible effects of attrition on effect estimates, and compute effect estimates that are adjusted for attrition without using imputed data. |
|
|
Term
| What are some deletion strategies for dealing with missing data? (Note: these techniques are not recommended) |
|
Definition
| Replacing dropouts with randomly selected persons drawn from the same pool, listwise deletion, pairwise deletion, and plugging in some sort of mean. |
|
|
Term
| What is listwise deletion? |
|
Definition
| Only analyzed subjects provide complete data. This deletion strategy applies to any statistical analysis and does not require additional statistical tests, but this is because less information is optimally used, making standard error increase, power decrease, and generalizability decrease. |
|
|
Term
| What is pairwise deletion? |
|
Definition
| Excluding based on the variable (cases with missing values). This deletion strategy is powerful, but underestimates variability. |
|
|
Term
| In deletion strategies, what does it mean to plug in some sort of mean? |
|
Definition
| Using the mean from elsewhere in the analysis to fill in missing data. This is not a good strategy and should be used as a last resort; it underestimates the true variability. |
|
|
Term
| What are the mechanisms of missingness? |
|
Definition
| Missing completely at random, missing at random, and missing not at random. |
|
|
Term
| What does it mean for data to be missing completely at random? |
|
Definition
| No pattern or cause is attributable to the missing data that originates from information in the data or how it was obtained. This produces unbiased estimators of population parameters, and is ignorable because it does not change the analyses/data/results. |
|
|
Term
| What does it mean for data to be missing at random? |
|
Definition
| It is missing because of something you assessed or didn't assess, but not the variable itself. This produces biased estimates of population parameters and is not ignorable. |
|
|
Term
| What does it mean for data to not be missing at random? |
|
Definition
| It is missing because of what you have assessed and related to a demographic. It produces biased estimates of population parameters and is not ignorable. |
|
|
Term
| What are some imputation methods that are used for missing data? |
|
Definition
| Hotdeck, multiple, dummy coding, and estimated maximum algorithm likelihood (EMA). |
|
|
Term
| What is hotdeck imputation? |
|
Definition
| An imputation strategy that identifies cases that are most similar to the one with missing values and randomly draws from among this group a participant value to replace the missing value. This is used by the census bureau and is simple, maintains the level of measurement, and completes the data at the end. However, the definition of "similar" is subjective. |
|
|
Term
| What is multiple imputation? |
|
Definition
| The process of replacing each missing data point with a set of m>1 plausible values to generate m complete datasets. These datasets are then analyzed by standardized statistical software, and the results combined, to give parameter estimates and standard errors that take into account the uncertainty due to missing data values. This is not usually used because mean substitution decreases the true variable estimate. It gives you a mean and standard error of the mean for each missing data and means of datasets can be combined to determine an overall effect. |
|
|
Term
|
Definition
| Substituting a constant for missing data. This creates an estimate that provides information for analysis but is a less accurate estimate of the variability. |
|
|
Term
| What is EMA (estimated maximum likelihood algorithm)? |
|
Definition
| Stepwise estimated value imputed mean based on other variables in the analysis (minus the bandwidth surrounding that value). This strategy is far superior to other strategies, but many outliers would skew the bandwidth surrouding the value. |
|
|
Term
| What are some things SPSS does in order to account for missing data? |
|
Definition
| It describes a pattern of missingness through univariate statistics (e.g., decreases in N mean that the data is not provided), estimating the mean, standard deviations, covariances, and correlations, or imputing values. |
|
|
Term
| What are some ways of estimating means, standard deviations, covariances, and correlations for missing data? |
|
Definition
| Listwise, pairwise, estimated maximization (EM) method, or regression. |
|
|
Term
| When estimating means, standard deviations, covariances, and correlations for missing data using a listwise approach, which types of cases are included? |
|
Definition
|
|
Term
| When estimating means, standard deviations, covariances, and correlations for missing data using a pairwise approach, what is required of the participants? |
|
Definition
| That they completed data on two variables. |
|
|
Term
| What is the estimated maximization (EM) method for estimating means, standard deviations, covariances, and correlations of missing data? |
|
Definition
| Condition probable of value given existing values. Expected values are temporarily substituted (estimated), maximum likelihood estimates are computed sa if the values are plugged into the missing spots (maximization). |
|
|
Term
| How is regression used in estimating means, standard deviations, covariances, and correlations of missing data? |
|
Definition
| Estimated value for missing participant using random effects of the coefficient. |
|
|
Term
| True or false: Missingness can be ignored. |
|
Definition
| False: missingness cannot be ignored. |
|
|
Term
| What does SPSS's missing variable analysis do to impute missing values? |
|
Definition
| It creates a new file (when run) with missing values imputed throughout. If done with estimated maximization, it was save the completed data. |
|
|
Term
| Give an example of a situation in which you may not want to impute a value. |
|
Definition
|
|
Term
| How does purposeful missing data occur on the part of the researcher? On the part of the participant? |
|
Definition
| Researchers may structure a survey with items like "If 'no' to #2, then skip to #6." A participant can purposefully miss data by refusing to answer questions or blindly circling the same answer throughout without reading the question. |
|
|
Term
| Values can be imputed when missingness is not related to... |
|
Definition
| Any characteristics of the person and is completely at random. |
|
|
Term
| The causal inference from any quasi-experiment requires that... |
|
Definition
| Cause precedes effect, cause covaries with effect, and alternative explanations for the causal relationship are implausible. |
|
|
Term
| How do quasi-experimental designs and randomized experiments compare? |
|
Definition
| Both manipulate treatment so cause precedes effect and both assess covariance statistically. However, randomized experiments rule out alternative explanations by distributing these alternatives across conditions, whereas quasi-experiments use alternative techniques. Quasi-experiments do not use random assignment. |
|
|
Term
| What are some stragies of quasi-experiments to rule out alternative causes? (Note: none offer the elegant statistical rationale of randome assignment) |
|
Definition
| Identify and study plausible threats to internal validity, control these threats by design (and statistically by covarying out the causal agents; design is better), and coherent pattern matching (specific predictions of patterns of results unlikely to occur by chance). |
|
|
Term
| What are the quasi-experimental designs without control groups? |
|
Definition
| One group posttest only, one group posttest only with multiple posttests, one group pretest posttest, one group pretest-posttest design using double pretest, one group pretest-posttest design using nonequivalent dependent variable, removed treatment design, and repeated treatment design. |
|
|
Term
| What are advantages/disadvantages of the one group posttest only design? |
|
Definition
Advantages: reduce plausibility of alternative explanations for treatment effects, yield unbiased estimates of the average treatment effects, and reasonable if you know a lot about the variable and the demographics of the population from which you have drawn.
Disadvantages: only internal validity threat it controls is selection bias, doesn't prevent/control things like regression to the mean, maturation, etc., and not a strong design. |
|
|
Term
| What are advantages/disadvantages of the one group posttest only with multiple posttests design? |
|
Definition
Advantage: allows for pattern matching.
Disadvantages: only internal validity threat it controls is selection bias, doesn't prevent/control things like regression to the mean, maturation, etc., not a strong design, adding multiple posttests can increase Type I errors, and caveat regarding pattern matching (cannot do posttest or look at the data first). |
|
|
Term
| What are advantages/disadvantages of the one group pretest posttest design? |
|
Definition
Advantage: adding the pretest provides some counterfactual information.
Disadvantages: counterfactual information is weak because of the many internal validity threats and threats to internal validity are a disadvantage (history, testing, attrition, etc.). |
|
|
Term
| What is an advantage to the one group pretest-posttest design using double pretest? |
|
Definition
| Reduces plausibility of maturation and regression. |
|
|
Term
| What occurs in a one group pretest-posttest design using nonequivalent dependent variable? |
|
Definition
| A changes with treatment and B does not, but they both measure similar constructs. |
|
|
Term
| What are advantages/disadvantages of the removed treatment design? |
|
Definition
Advantages: good example of coherent pattern matching and outcome should rise/fall with presence/absence of treatment.
Disadvantages: treatment effect has to dissipate quickly and there can be some ethical issues. |
|
|
Term
| What are advantages/disadvantages of the repeated treatment design? |
|
Definition
Advantages: very few threats to internal validity could explain this pattern (treatment would have to covary with introduction/removal of treatment, which is unlikely) and it is good with transient effects or unobtrusive treatment or long intervals.
Disadvantage: possible threat to internal validity - cyclical maturation (unfortunately, this type of design is easy to pair with cyclical patterns). |
|
|
Term
| What are the quasi-experimental designs with a control group but no pretest? |
|
Definition
| Posttest only with nonequivalent groups, posttest only design using an independent pretest sample, and posttest only using proxies for pretests. |
|
|
Term
| What is the posttest only with nonequivalent groups design? |
|
Definition
| A design that adds a control group to the one group posttest only design. This is used when you're called in late. This is used if the pretest would have a sensitizing effect. |
|
|
Term
| What are advantages of the posttest only design using an independent pretest sample? |
|
Definition
| It is useful when pretest measures are reactive, when it is too difficult or expensive to do a longitudinal study, or when one wishes to study intact communities whose members change over time. It also draws its second sample from the same population as the treatment group. |
|
|
Term
| What are some characteristics of the posttest only design using proxies for pretests? |
|
Definition
| Variables are conceptually related to and correlated with posttest within treatments. Preferably, these proxies should be conceptually related to the outcome, not just readily accessible measures (e.g., age, race, etc.). Matching procedures are also used. |
|
|
Term
| What are the matching procedures used for the posttest only design using proxies for pretests? |
|
Definition
| Exact (same score, twin, etc.), caliper (matched person falls within bandwidth of other persons selected), index, and benchmark (examples of propensity matching, select controls close to treatment group based on multivariate distance). |
|
|
Term
| What are the quasi-experimental designs with a control group and pretest? |
|
Definition
| Untreated control group with dependent pretest and posttest samples, untreated control group with dependent pretest and posttest using double pretest, untreated control group with dependent pretest and posttest using switching replications, untreated control group with dependent pretest and posttest using reversed treatment control group, and cohort designs. |
|
|
Term
| What are some characteristics of the untreated control group with dependent pretest and posttest samples design (also called nonequivalent comparison group design)? |
|
Definition
| Pretest/comparison group examine threats to validity easier. Groups are nonequivalent by definition and selection bias is assumed present. Pretest - magnitude assessment, selection bias direction, attrition, no pretest - doesn't mean selection bias is not present. When pretest differences, selection may be combined with other threats. |
|
|
Term
| What are some graphical representations of the untreated control group with dependent pretest and posttest sample design? |
|
Definition
| Both groups grow apart in the same direction, no changes in control group, initial pretest differences favoring treatment group diminish over time, and outcomes cross over in the direction of relationships. |
|
|
Term
| What are some characteristics of the untreated control group with dependent pretest and posttest samples design when both groups grow apart in the same direction (fan-spread interaction)? |
|
Definition
| If group mean differences are a result of the selection-maturation threat, then differential growht between groups should be occurring within groups. Test this by a series of within-group analyses. Selection maturation threat is associated with posttest within-group variances that are greater than the corresponding greatest variances. Plot pretest scores against hypothesized maturational variable for the experimental and control groups separately. If regression lines differ, differential growth rates are likely. |
|
|
Term
| What are some of the characteristics of the untreated control group wtih dependent pretest and posttest samples design when there are no changes in the control group? |
|
Definition
| The critic must explain why spontaneous growth occurred only in the treatment group and sometimes, within-group analyses can shed light on such between-group threats. For example, treatment group matured faster because they were older, in the treatment group, divided people based on age. |
|
|
Term
| What are some characteristics of the untreated control group with dependent pretest and posttest samples design when initial pretest differences favoring the treatment group diminish over time? |
|
Definition
| This is a characteristic pattern hypothesized in compensatory programs. Outcome is subject to typical scaling (selection-instrumentation) and history (selection-history) threats. It is important when you have this type of finding to thoroughly investigate the reasons for the initial differences. |
|
|
Term
| What are some characteristics of the untreated control group with dependent pretest and posttest samples design when outcomes cross over in the direction of relationships? |
|
Definition
| The pattern is particularly amenable to causal interpretations. The plausibility of selection-maturation is reduced becuase it is the best interaction to have. Selection-maturation threats are less likely because crossover interaction maturation patterns are not widely expected. The outcome renders a regression threat is less likely. The caveat to this is the power to detect a statistically reliable interaction is low. |
|
|
Term
| What are the characteristics of the untreated control group with dependent pretest and posttest using double pretest design? |
|
Definition
| The double pretest allows the researcher to understand possible biases in the main treatment analysis and permits assessment of selection-maturation threat on the assumption that rates between O1 and O2 will continue between O2 and O3. This assumption is only testable for the untreated group. It also allows for an analysis of regression effects. However, within-group rates will be fallibly estimated (measurement error) and instrumentation shifts could make measured growth between O1 and O2 unlike that between O2 and O3. |
|
|
Term
| What are the characteristics of the untreated control group with dependent pretest and posttest using switching replications design? |
|
Definition
| Strong design but 2nd phase is not exact replication both historically and because treatment was removed from the 1st group - second introduction of the treatment is best be thought of as a modified replication that probes the internal and external validity of whether this new context changes the treatment effect. Only a pattern of change that mimics the sequence of treatment introductions can serve as an alternative interpretation. This design makes very clear predictions about treatment effects. |
|
|
Term
| What are characteristics of the untreated control group with dependent pretest and posttest using reversed treatment-control group design? |
|
Definition
| X+ represents treatment expected to produce an effect in one direction and X- represents a conceptually opposite treatment effect. This design has special construct validity advantages. |
|
|
Term
| What are the cohort designs? |
|
Definition
| Cohort control group design and cohort control group design with pretest from each group. |
|
|
Term
|
Definition
| Successive groups that go through a process. |
|
|
Term
| What are characteristics of cohort designs? |
|
Definition
| A critical assumption with cohorts is that selection differences are smaller between cohorts than would be the case between non-cohort comparison groups. |
|
|
Term
| When are cohorts useful as control groups? |
|
Definition
| One cohort experiences a given treatment and earlier or later cohorts do not, cohorts differ in only minor ways from their contiguous cohorts (IMPORTANT), organizations insist that a treatment be given to everybody, thus precluding simultaneous controls making possible only historical controls, an organization's archival records can be used for constructing and comparing cohorts, and it is important to assume that cohorts together are fairly similar. |
|
|
Term
| How do cohort control group design and cohort control group design with pretest from each group compare? |
|
Definition
| Both are examples of cohort control group designs and history is a threat in both designs. In cohort control group design with pretest from each group, there is a math advantage allowing for better assessment of maturation and regression. Also, this design measures proxies for pretests and these variables are conceptually related to and correlated with the posttest within treatments. |
|
|
Term
| What are the types of quasi-experiments? |
|
Definition
| Quasi-experiments with no control groups, quasi-experiments with no pretest, and quasi-experiments with both controls and pretests. |
|
|
Term
| What are the contrasts that can be done if you do not have a control group? |
|
Definition
| Regression extrapolation contrasts, normed comparisons contrasts, and secondary source contrasts. |
|
|
Term
| What are characteristics of regression extrapolation contrasts? |
|
Definition
| Compare obtained posttest scores of tx group with the score that would've been predicted based on other information. Assumes reliable estimates of true scores. W/O full knowledge of threats to validity, predicted scores rarely yield valid counterfactual inferences. Depends on stable estimates estimation using reliable measures and large samples. Only worth doing when no other control group is possible or adjunct another procedure. |
|
|
Term
| What are characteristics of normed comparison contrasts? |
|
Definition
| Performance of the treatment group at pretest and posttest is compared with available published norms that shed light on counterfactual inferences. Normed contrast provides weak counterfactual information and the comparison is also threated by selection, history, testing, regression, and maturation. Sometimes, you can resolve some of these concerns by using "local" normative samples. |
|
|
Term
| What are characteristics of secondary source contrasts? |
|
Definition
| Construct opportunistic contrasts using secondary sources, such as records of cases treated prior to the advent of the new treatment. The use of such archival or history data is challenging practically and some of the same conceptual issues that are problematic of normative contrasts apply here. It is the most common contrast and easy to examine threats of validity. There is no random assignment, so there is a selection bias. Look for attrition reason in pretest. |
|
|
Term
| In a general sense, what are ethics? |
|
Definition
| What is best for the individual and society. |
|
|
Term
| True or false: ethics can be situational or universal. |
|
Definition
|
|
Term
| A common question regarding ethics may be is what is right or wrong a function of... |
|
Definition
|
|
Term
| What are the three types of ethics commonly identified? |
|
Definition
| Metaethics, normative ethics, and applied ethics. |
|
|
Term
| What are some characteristics of metaethics and common questions it considers? |
|
Definition
| What does ethical mean? Does it exist? Do ethics represent universal truths? Moral absolute - religious code (morals are not equivalent to ethics). The big questions, existential and epidemiological concerns. Immoral contradiction of principles and moral attitudes, ethical codes. |
|
|
Term
|
Definition
| A universal set of rules adhered to by a professional group. These do not recognize metaphysical entity. |
|
|
Term
| What are characteristics of normative ethics? |
|
Definition
| Bridge the gap between metaethics and applied ethics. Practical moral standard, theory of conduct, and theory of value. |
|
|
Term
| What are practical moral standards? |
|
Definition
| Arrive at moral standards telling us right from wrong and how to live life (basically, normative ethics). |
|
|
Term
| What is theory of conduct? |
|
Definition
| Knowing the difference between right and wrong and what our obligations are. |
|
|
Term
|
Definition
| Determine if there is value in anything, including our decisions - intrinsically good. |
|
|
Term
| What are characteristics of applied ethics? |
|
Definition
| Ethics specifically applied, like normative ethics applied to controversial issues. |
|
|
Term
| What role did the Nazi doctors play in the development of the 20th century ethical codes? |
|
Definition
| The behavior of Nazi doctors and the wide range of appalling experiments done in the name of medical science (e.g., high altitude experiments, incendiary bomb experiments, experiments in freezing water, and experiments on women and children) began the development of ethical codes. |
|
|
Term
| What happened in Nuremberg? |
|
Definition
| 23 Nazi doctors were tried for murders and crimes. 16 were found guilty and 7 were sentenced to death. |
|
|
Term
| What does the Nuremberg Code (1947) state regarding ethical practices that laid the groundwork for our ethical guidelines? |
|
Definition
| Participants must have the legal capacity to give consent, exercise free power of choice, no element of force, nature/duration/purpose of the experiment should be known, all inconveniences and hazards should be explained, duty/responsibility for disclosing information rests on the person initiating research, your experiment should yield results that are good for society that cannot be obtained another way, and risk has to be justified. |
|
|
Term
| Some examples of unethical studies? |
|
Definition
| Tuskegee Syphilis study, Milgram study, Belmont Report. |
|
|
Term
| What are the three basic principles of the Belmont Report? |
|
Definition
| Respect for persons, beneficence, and justice. |
|
|
Term
| According to Bersoff (1994), what are some recruitment issues that may exist and some things IRB focuses on with regard to issues in research design? |
|
Definition
| Recruitment issues include hyperclaiming and causism. IRB may ask the question, "If no fruits are produced, how can the means be greater than the end?" |
|
|
Term
| What are some institutional safeguards IRB considers in the Policy for Protection of Human Subjects (U.S. Department of Health and Human Services)? |
|
Definition
| Required, diversity, scientific/nonscientific base, independence. |
|
|
Term
| What is the criteria for IRB approval? |
|
Definition
| Risks to subjects minimized/reasonable, equitable selection of subjects, informed consent/appropriate documentation, data monitoring, and privacy of subjects. |
|
|
Term
| What are the requirements for informed consent? |
|
Definition
| Statement of purpose/explanation, description of procedures, forseeable risks, benefits, alternative procedures, statement of confidentiality, compensation if more than minimal risk, voluntary refusal, and contact person. |
|
|
Term
| What are some issues surrounding informed consent? |
|
Definition
| What exactly is "informed" consent? Drug trials imply knowledge of placebo condition for informed consent. The irony of providing too much detail can elicit paranoid ideation or become too difficult for a participant to read. |
|
|
Term
| Research involving more than minimal risk is approved contingent upon... |
|
Definition
| Anticipated benefit (absolute benefit and relative to alternative treatments) and generalizability of benefit. |
|
|
Term
| What are some critiques of IRBs? |
|
Definition
| Wield a lot of power, overly cautious/conservative (trade-off of scientific advances for excessive ethical concerns), appropriate role (humane treatment of subjects versus watchdog of methodology), politics and research (sociopolitical implications), and APA's lack of enforcement. |
|
|
Term
| Key rational for animal research? |
|
Definition
| CARE policy statement (APA's Committee on Animal Research and Ethics): animal research advances animal and human welfare. Psychologists do research to learn more about behavior and how knowledge of behavior can be used to advance the welfare of people and animals. Identification of characteristics unique to different species has yielded information that contributes to understanding and advancing the welfare of animals and people. |
|
|
Term
| How has animal research contributed significantly to our knowledge of behavior? |
|
Definition
| Knowledge of beasic learning processes/motivational systems (hunger, thirst, reproduction). Critical info about sensory processes (vision, taste, hearing, pain perception). Connections between stress and disease. Suggested psychological interventions for coping with stress more effectively. Understanding of drug abuse/physical dependence. Critical to efforts to develop effective pharmacologic treatments for drug dependence and cognitive deficits of aging/Alzheimer's disease. |
|
|
Term
| How has animal research helped to explain the central nervous system? |
|
Definition
| Experiences in the world shape behavior, understanding how nervous system works critical to complete understanding of behavior, including behaviors problematic in mental illness, memory disorders, drug addictions. Process of recovery after neural damage: (1) Biological correlates of fear, anxiety, and other forms of stress, (2) Subjective and dependence-producing effects of psychotropic drugs, and (3) Mechanisms that control eating and other motivational processes. |
|
|
Term
| What are some human subject projects and alternatives to live subjects that have been proposed? |
|
Definition
| Use of plants/tissue cultures or computer simulations (lack central nervous systems). All who do research with animals are required, legally and ethically, to consider the possibility of using alternatives to nonhuman animals (e.g., observing animals in natural environment - psychologists observe/study animals in natural environments; for many investigations, the lab is only setting in which causal variables can be isolated with sufficient precision to generate meaningful conclusions. |
|
|
Term
| How do animals in psychology research vary? |
|
Definition
| 7-8% og psychology research involves the use of animals. 90% of animals used have been rodents and birds, primarily rats, mice, and pigeons. 5% of the animals are monkeys and other primates. Use of dogs or cats is rare. |
|
|
Term
| What are some things that are done to be sure that humane care and use of animals in research is ensured? |
|
Definition
| Many safeguards exist to assure that laboratory animals receive humane and ethical treatment. Care and use of animals in research are regulated and monitored by various government agencies. Institutions in which research with animals is conducted have federally mandated review committees. |
|
|
Term
| What are some federal regulations and guidelines that exist for using animals in research? |
|
Definition
| Animal Welfare Act (most recently amended, 1985) governs the care and use of many research animals. The U.S. Department of Agriculture is responsible for enforcing the act and conducting periodic unannounced inspections of animal research facilities, both public and private. Institutions that conduct research using animals covered by the act are required to have an Institutional Animal Care and Use Committee (IACUC) to review each research protocol. |
|
|
Term
| How does the scientific community set standards with regards to the use of animals in research? |
|
Definition
| The American Association for the Accreditation of Laboratory Animal Care (AAALAC) is nationally and internationally recognized for its institutional accreditation program. It sets the "gold standard" for laboratory animal care and serves as a guide for those research facilities seeking to assure the best conditions for their research animals. Once accreditation is obtained, thorough inspections are conducted every 3 years to determine whether an institution may retain its accreditation. |
|
|
Term
| What are the APA ethics code and other guidelines that cover treatment of research animals? |
|
Definition
| APA Ethical Principles of Psychologists/Code of Conduct includes principles for humane/ethical treatment of research animals. APA members are committed to uphold principles. Failure to do so can lead to expulsion from association. APA's Guidelines for Ethical Conduct in the Care and Use of Animals sets comprehensive standards for psychologists using animals in research. Those who publish in APA journals are required to conduct research with animals in accordance with guidelines. |
|
|
Term
| What are some key principles of the new APA guidelines (2003) for animal research? |
|
Definition
| Acquire/care for/use/dispose of animals in compliance w/federal/state/local laws/regulations w/professional standards. Ensure those under supervision using animals received instruction in research methods and in care/maintenance/handling of species, to extent appropriate to role. Reasonable efforts to minimize discomfort/infection/illness/pain of animals. Use procedure subjecting animals to pain/stress/privation only when alternative unavailable and goal is justified by prospective scientific/educational/applied value. Perform surgery under appropriate anesthesia/follow techniques to avoid infection/minimize pain during/after. Proceed rapidly when appropriate that an animal's life be ended, w/effort to minimize pain in accordance with procedures. |
|
|
Term
| What was the Tuskegee Syphilis study and what role did it play in research ethics? |
|
Definition
| Longitudinal study (1932-72) of syphilis. Tx for syphilis available in 1947 but researchers never told participants they had syphilis nor did they tx them for it. Participants given pink aspirin. No informed consent. Participants uneducated/illiterate. 399 black participants, 28 died of syphilis, 100 died of other complications, 40 women infected by participants, and 19 children born with syphilis. Led to regulations of research ethics being enforced and development of Belmont Report. |
|
|
