Term
|
Definition
1. does the test measure what it's supposed to measure?
2. accuracy of inferences for a given purpose
3. a matter of degree; not all-or-none
4. is it sufficient to make use of the test worthwhile? |
|
|
Term
|
Definition
| whether a test looks reasonable to the test-taker |
|
|
Term
|
Definition
matching test items to a well-defined universe of content
typically used in educational achievement tests and job-related tests
often uses a "test blueprint" from curricular materials
use of "Bloom's Taxonomy" |
|
|
Term
| Criterion-related validity |
|
Definition
relation of the test to an external criterion
ex: SAT as a predictor of college GPA
three common approaches: external, realistic behavioral criterion; group contrasts; scores on another test
concurrent; predictive |
|
|
Term
|
Definition
| whether a test measures if it's supposed to measure, and only what it is supposed to measure |
|
|
Term
| Construct-irrelevant variance |
|
Definition
| when a test measures characteristics other than what it is meant to measure |
|
|
Term
| Concurrent criterion-related validity |
|
Definition
| when there is agreement between test performance and current status on some other variable |
|
|
Term
| Predictive criterion-related validity |
|
Definition
| when a test aims to predict status on some criterion that will be attained in the future |
|
|
Term
| External criterion approach |
|
Definition
| provides a realistic definition of the construct of interest |
|
|
Term
|
Definition
| compare two groups that you suspect will get different scores |
|
|
Term
|
Definition
| performance on the test influences status on the criteria; rater assigns score based on scores on a test, not on his or her own judgment; |
|
|
Term
|
Definition
| a relatively high correlation between the test and some criterion thought to measure the same construct as the test |
|
|
Term
|
Definition
| linearity (nonlinear- r will underestimate the true degree of the relationship), group homogeneity (study conducted on a heterogeneous group when examining a homogeneous group), heteroscedasticity (the assumption that points are equally scattered around the prediction line throughout the range) |
|
|
Term
| Reliability-validity relationship |
|
Definition
| reliability is necessary, but not sufficient for a test to be valid |
|
|
Term
|
Definition
a set of methods for conceptualizing and analyzing the statistical properties of items and tests
the performance of an examinee on a test item can be predicted or explained by a set of factors called traits, latent traits, or abilities |
|
|
Term
|
Definition
number of items (per score): 15-20
p-values: even distribution for each item
discrimination: relation of p and D - IMPORTANT
content considerations: get right content |
|
|
Term
|
Definition
purpose to develop norms
key factor is representativeness of the group |
|
|
Term
|
Definition
| ability to think abstractly, identify relationships, memory functions, learn from experience, deal effectively with symbols |
|
|
Term
| Spearman's factor analysis |
|
Definition
two factors: "g" and "s"; emphasis on "g"
origin: study of correlations |
|
|
Term
|
Definition
|
|
Term
|
Definition
prototype for many multifactor intelligence tests
usual: spatial, numerical, verbal, reasoning
other: memory, perceptual, fluency |
|
|
Term
|
Definition
another extreme multi-factor model of intelligence
largely discredited on methodological grounds
influential re: divergent (creative) thinking |
|
|
Term
| Cattel and Horn's g-f and g-c |
|
Definition
crystallized: developed by experience, etc,
fluid: raw potential (how to measure?) |
|
|
Term
|
Definition
| an information processing model by Das, et al. |
|
|
Term
|
Definition
| a trait or characteristic |
|
|
Term
| Construct underrepresentation |
|
Definition
| when the construct of interest is not fully covered by the test |
|
|
Term
|
Definition
| whether that content on a test has actually been taught in class |
|
|
Term
|
Definition
| the process of developing the list of knowledge and skills required by a job |
|
|
Term
|
Definition
| area of overlap of test and construct |
|
|
Term
|
Definition
| a correlation coefficient used to express test validity |
|
|
Term
|
Definition
| shows that a test has a relatively low correlation with constructs other than the construct it is intended to measure |
|
|
Term
| Multitrait-multimethod matrix |
|
Definition
| a correlation matrix including tests that purport to measure many different traits using many different methods; helps demonstrate that correlations within a trait but cutting across different methods are higher than correlations within methods but cutting across different traits |
|
|
Term
| Selectivity (sensitivity) |
|
Definition
| correctly identifying those who have the problem (the criterion group) |
|
|
Term
|
Definition
| correctly identifying those who do not have the problem (the contrast group) |
|
|
Term
|
Definition
|
|
Term
|
Definition
| a family of statistical techniques that help to identify the common dimensions underlying performance on many different measures |
|
|
Term
|
Definition
| the study of how examinees go about responding to a test |
|
|
Term
|
Definition
| refers to the consequences of test use and interpretation |
|
|
Term
|
Definition
| test may be more valid for one subgroup than another |
|
|
Term
|
Definition
| increase in validity achieved by adding a new predictor (e.g. ACT) to existing ones |
|
|
Term
|
Definition
| degree to which validity evidence generalizes to other settings |
|
|
Term
|
Definition
| the percentage of individuals in the population having some characteristic |
|
|
Term
|
Definition
| a technique for summarizing the actual statistical information contained in many different studies on a single topic |
|
|
Term
|
Definition
|
|
Term
|
Definition
| MC, T/F, forced-choice, likert scale; scored either correct/incorrect or in a certain direction |
|
|
Term
| Constructed response (free response) |
|
Definition
| essay, performance, portfolio; oral, projective; inter-scorer reliability is an issue |
|
|
Term
|
Definition
| an item's ability to differentiate statistically in a desired way between groups of examinees |
|
|
Term
|
Definition
| an item's ability to differentiate statistically in a desired way between groups of examinees |
|
|
Term
|
Definition
| relates performance on an item to status on the trait or ability underlying the scale |
|
|
Term
| Differential item functioning |
|
Definition
| addresses the question of whether test items function differently for different groups of examinees for reasons other than actual differences on the the trait being measured |
|
|
Term
|
Definition
| trying out items selected from item tryout on a second sample to further validate them |
|
|
Term
|
Definition
| loss invalidity of test items after cross-validation |
|
|
