Term
|
Definition
| measuring what we are supposed to be measuring |
|
|
Term
|
Definition
| the validity at face value, ie a good translation of the construct |
|
|
Term
|
Definition
| measurement covers the full range of the concept |
|
|
Term
|
Definition
| measure predicts scores on other things |
|
|
Term
|
Definition
| ability to generalize to your construct (eg, self-esteem scales) |
|
|
Term
|
Definition
| different items are measuring the same underlying construct |
|
|
Term
|
Definition
| the approxiamte truth about inference regarding cause and effect |
|
|
Term
| parallel-forms reliability |
|
Definition
| Used to assess the consistency of the results of two tests constructed in the same way from the same content domain. create 2 parallel forms- large set of questions that address the same construct and randomly divide into two sets. You administer both instruments to the same sample of people. The correlation between the two parallel forms is the estimate of reliability |
|
|
Term
|
Definition
| a test correlates well with a test of the same thing that has been previously validated |
|
|
Term
|
Definition
| A measure of consistency where a test is split in two and the scores for each half of the test is compared with one another. If the test is consistent it leads the experimenter to believe that it is most likely measuring the same thing. This is not to be confused with validity where the experimenter is interested if the test measures what it is suppose to measure. A test that is consistent most likely is measuring something; the experimenter just does not know what that "something" is. This is why it is said that reliability sets the ceiling of validity. |
|
|
Term
|
Definition
| bias that occurs because when we know that other people are watching us, we will tend to behave in a way we believe is socially acceptable and desirable. |
|
|
Term
|
Definition
can be applied to both the way we use various different methods in the research process ("methodological triangulation") and the way we can combine various theoretical perspectives in our research ("theoretical triangulation"). It might, therefore, be useful to look at each of these briefly in turn. three basic types of methodological triangulation:
a. One researcher using two or more research techniques.
b. Two or more researchers using the same research technique.
c. Two or more researchers using two or more research techniques.
In general terms, methodological triangulation can be used for a number of purposes:
* To collect different types of information (qualitative and quantitative, primary and secondary, for example).
* With two or more researchers using the same method (observation, for example) their observations can be compared to see if they agree that they have seen the same things in the same ways.
* To check that data collected in one form (for example, through a structured interview) is both reliable and valid. As I've already noted, some form of observational method could be used to check these things.
* To verify (that is, "confirm") that any data collected is accurate. |
|
|
Term
| Validity and Reliability in "Measures" (12 ways) |
|
Definition
| content validity, predictive validity, construct validity, internal consistency, internal validity, parallel forms reliability, Cronbach's alpha, criterion validity, concurrent validity, split-half reliability, social desirability, data trangulation |
|
|
Term
| Interobserver reliability |
|
Definition
| observers consistent in observations: Whenever you use humans as a part of your measurement procedure, you have to worry about whether the results you get are reliable or consistent. People are notorious for their inconsistency. We are easily distractible. We get tired of doing repetitive tasks. We daydream. We misinterpret. There are two major ways to actually estimate inter-rater reliability. If your measurement consists of categories -- the raters are checking off which category each observation falls in -- you can calculate the percent of agreement between the raters.he other major way to estimate inter-rater reliability is appropriate when the measure is a continuous one. There, all you need to do is calculate the correlation between the ratings of the two observers. |
|
|
Term
|
Definition
| natural tendency of any sample, especially a small one, to differ from its population (also known as the null hypothesis). |
|
|
Term
|
Definition
| A term referring to the tendency of some people to work harder and perform better when they are participants in an experiment. Individuals may change their behavior due to the attention they are receiving from researchers rather than because of any manipulation of independent variables. |
|
|
Term
| Reliability and validity in "researchers" (4 ways) |
|
Definition
| interobserver reliability, sampling error, hawthorne effect, social desirability |
|
|
Term
|
Definition
| subjects cover the full range of the concept being studied. Check operationalization against relevant content domain for the construct |
|
|
Term
|
Definition
| We estimate test-retest reliability when we administer the same test to the same sample on two different occasions. This approach assumes that there is no substantial change in the construct being measured between the two occasions. The amount of time allowed between measures is critical. We know that if we measure the same thing twice that the correlation between the two observations will depend in part by how much time elapses between the two measurement occasions. The shorter the time gap, the higher the correlation; the longer the time gap, the lower the correlation. This is because the two observations are related over time -- the closer in time we get the more similar the factors that contribute to error. Since this correlation is the test-retest estimate of reliability, you can obtain considerably different estimates depending on the interval. |
|
|
Term
|
Definition
| amount of treatment effect that can be attributed to having taken the pretest measure |
|
|
Term
| selection-history interaction |
|
Definition
| any event that occurs between pretest and posttest that the groups experience differently |
|
|
Term
|
Definition
| ability to generalize to the larger population |
|
|
Term
|
Definition
| the measurable, observable, or felt improvement in health or behavior not attributable to a medication or invasive treatment that has been administered. |
|
|
Term
|
Definition
| a type of test, assessment, or evaluation which yields an estimate of the position of the tested individual in a predefined population, with respect to the trait being measured. This estimate is derived from the analysis of test scores and possibly other relevant data from a sample drawn from the population. [1] The term normative assessment refers to the process of comparing one test-taker to his or her peers.[1] Norm-referenced assessment can be contrasted with criterion-referenced assessment and ipsative assessment. |
|
|
Term
|
Definition
| The effect on the test variable, caused by a variable external to an experiment, which is associated with a change in the measurement instrument. Did anything happen to the technical environment or the measurement tools or instrumentation that could significantly influence results? |
|
|
Term
|
Definition
a statistical bias in which there is an error in choosing the individuals or groups to take part in a scientific study.[1] It is sometimes referred to as the selection effect. The term "selection bias" most often refers to the distortion of a statistical analysis, resulting from the method of collecting samples. If the selection bias is not taken into account then any conclusions drawn may be wrong.
(1) self-selection of individuals to participate in an activity or survey, or as a subject in an experimental study; (2) selection of samples or studies by researchers to support a particular hypothesis. |
|
|
Term
|
Definition
| describes the above-average performance of a control group placed in competition with an experimental group |
|
|
Term
| Validity and reliability in "subjects" (11 ways) |
|
Definition
| content validity, test-retest reliability, pretest sensitization, selection-history interaction, external validity, placebo effect, norm-referenced tests, instrumentation effects, selection bias, hawthorne effect/ John Henry effect, Social desirability |
|
|
Term
|
Definition
| certain behaviors change naturally with age and time |
|
|
Term
|
Definition
| asking people when things happen is susecptible to memory lapses. Forward telescoping- people think events occured more recently than they did. Backward telescoping- other events occured more recentnly than they think |
|
|
Term
| Relaibility and Validity with "time" (7 ways) |
|
Definition
| maturation trends, instrumentation effects, predictive validity, test-restest relaibility, selection history interaction, criterion validity, telescoping |
|
|
