Term
|
Definition
| assignment of numbers to differentiate values of a variable |
|
|
Term
|
Definition
| procedures for collecting information and using it to make decisions for which some value is placed on the results |
|
|
Term
|
Definition
- multiple meanings
a.Measurement of a variable
b.Evaluation
c.Diagnosis of individual difficulties
d.Procedures to gather information on student performance |
|
|
Term
| Define the terms measurement, evaluation, and assessment. Differentiate one from the other |
|
Definition
1.Measurement: assignment of numbers to differentiate values of a variable
2.Evaluation: procedures for collecting information and using it to make decisions for which some value is placed on the results
3.Assessment - multiple meanings
a.Measurement of a variable
b.Evaluation
c.Diagnosis of individual difficulties
d.Procedures to gather information on student performance |
|
|
Term
| Identify two reasons why measurement is a critically important component of quantitative research. |
|
Definition
1.Obtain information about the variables being studied
2.Provide a standard format for recording observations, performances, or other responses of subjects
3.Provide for a quantitative summary of the results from many subjects |
|
|
Term
| Differentiate the four measurement scales and provide educationally relevant examples of each. |
|
Definition
Nominal - categories
ie:
Race
Gender
Types of schools (e.g., public, private, parochial)
Ordinal - ordered categories
ie:
Finishing position in a race
Ranks in the military
Grade levels
Interval - equal intervals between numbers on the scale
ie:
Test scores
Achievement levels
Ratio - equal intervals and an absolute zero (0)
ie:
Height
Weight
Time |
|
|
Term
| Define the term descriptive statistics |
|
Definition
| statistical procedures that summarize a set of numbers in terms of central tendency, variation, or relationships |
|
|
Term
| Describe the characteristics of a frequency distribution |
|
Definition
an organization of the data set indicating the number of times (i.e., frequency) each score was present
Types of presentations:
Frequency table
Frequency polygon
Histogram
Shapes of distributions: |
|
|
Term
| Describe the characteristics of a normal distribution, a positively skewed distribution, and an negatively skewed distribution |
|
Definition
Normal - a set of scores that are equally distributed around a middle score (i.e., the mean)
Positively skewed - a set of scores characterized by a large number of low scores and a small number of high scores
Negatively skewed - a set of scores characterized by a large number of high scores and a small number of low scores
See Figure 6.2 |
|
|
Term
| Explain the concept of central tendency and describe the characteristics of the mode, median, and mean as measures of central tendency. |
|
Definition
the typical score
Mode: the most frequently occurring score
Median: the score above and below which one-half of the scores occur
Mean
1. The arithmetic average of all scores
2. Statistical properties make it very useful
3. Concerns related to outlying scores |
|
|
Term
| Explain the concept of variation and describe th characteristic of the range and standard deviation as measures of variation |
|
Definition
how different are the scores
Range: the difference between the highest and lowest scores
Standard deviation:
The average distance of the scores from the mean
The relationship to the normal distribution
±1 SD 68% of all scores in a distribution
±2 SD 97% of all scores in a distribution
Use of percentile ranks - the percentage of scores at or below a specified score
See Figure 6.4 |
|
|
Term
| Explain the relationship between the standard deviation and the normal curve. |
|
Definition
Standard deviation:
The average distance of the scores from the mean
The relationship to the normal distribution
±1 SD 68% of all scores in a distribution
±2 SD 97% of all scores in a distribution
Use of percentile ranks - the percentage of scores at or below a specified score
See Figure 6.4 |
|
|
Term
| Explain the concept of relationship and describe the characteristics of the correlation coefficient as a measure of relationship. Interpret correlation coefficients in terms of direction and strength. |
|
Definition
Correlation
A measure of the relationship between two variables
Strength - 0.00 to 1.00
0.00 indicates no relationship and consequently no predictability
1.00 indicates perfect relationship and consequently perfect predictability
Direction - positive (+) or negative (-)
Positive: high and low scores on the first variable are related to high and low scores respectively on the second variable
Negative: high and low scores on the first variable are related to low and high scores respectively on the second variable
Scatterplots - visualizations of correlations
See the following web site for an interactive scatterplot demonstrating varying levels of correlations |
|
|
Term
| Define validity as it relates to educational measures |
|
Definition
| Validity: the extent to which inferences are appropriate, meaningful, and useful |
|
|
Term
| identify five characteristics of validity |
|
Definition
Refers to the interpretation of the results
A matter of degree
Specific to a particular use or interpretation
A unitary concept
Involves an overall evaluative judgment |
|
|
Term
| explain the effect of validity on research |
|
Definition
1. If the research results are to have any value, validity of the measurement of a variable must exist
a. Use of established and "new" instruments and the implications for establishing validity
b. Importance of establishing validity prior to data collection (e.g., pilot tests)
2. Validity as a matter of degree (i.e., the extent to which...)
3. Judged on the basis of available evidence
4. Varying levels of validity evidence are reported in articles |
|
|
Term
| Identify three sources of validity evidence and give an example of each. |
|
Definition
1. Test content - evidence of the extent to which items on a test are representative of the larger domain of content or items from which they are drawn
2. Internal structure - evidence of the extent to which the relationships between items and parts of the instrument are consistent with those reflected in the theoretical basis of the instrument or its intended use
3. Relationships with other variables - evidence of the extent to which scores from an instrument are related to similar as well as different traits
a. Convergent evidence - scores correlate with measures of the same thing being measured
b. Discriminate evidence - scores do not correlate with measures of something different than that being measured
c. Predictability - the extent to which test scores predict performance on a criterion variable |
|
|
Term
| Define reliability of measurement as it relates to educational measures |
|
Definition
| the extent to which scores are free from error |
|
|
Term
| identify several sources of measurement error |
|
Definition
(see Table 6.3, p. 138)
1.Test construction and administration (e.g., ambiguous questions, confusing directions, changes in scoring, interrupted testing, etc.)
2.Subject's characteristics (e.g., test anxiety, lack of motivation, fatigue, guessing, etc.) |
|
|
Term
| explain the effect of reliability of research |
|
Definition
1.If the results are to have any value, reliability of the measurement of a variable must exist
a. Established prior to conducting the research (e.g., pilot study)
b. Necessary but not sufficient condition for validity (i.e., to be valid, an instrument must be reliable, but a reliable instrument is not necessarily valid)
2. Conditions affecting reliability
a. Length of the test (i.e., longer tests are typically more reliable)
b. Subjects
1.Instruments used with heterogeneous samples typically have higher reliability than those used with homogeneous samples
2.Scores for older subjects are typically more reliable than those for younger children
c. Trait being measured (i.e., cognitive traits are more reliable than affective characteristics)
3.Enhancing reliability
a. Standardized administration procedures (e.g., directions, conditions, etc.)
b. Appropriate reading level
c. Reasonable length of the testing period
d. Counterbalancing the order of testing if several tests are being given |
|
|
Term
| Identify five types of reliability estimates and give an example of each. |
|
Definition
1.Stability (i.e., test-retest)
a. Testing the same subject using the same test on two occasions
b. Limitation - carryover effects from the first to second administration of the test
2.Equivalence (i.e., parallel form)
a. Testing the same subject with two parallel (i.e., equal) forms of the same test taken at the same time
b. Limitation - difficulty in creating parallel forms
3.Equivalence and stability
a. Testing the same subject with two forms of the same test taken at different times
b. Limitation - difficulty in creating parallel forms
4.Internal consistency
a. Testing the same subject with one test and "artificially" splitting the test into two halves
b. Forms
1.KR 20
a) Dichotomously scored items (i.e., right or wrong)
b) Typical of cognitive measures
2.Cronbach alpha
a) Non-dichotomously scored items (e.g., strongly agree, agree, disagree, strongly disagree)
b) Typical of non-cognitive measures
c. Limitations - must have a minimum of ten (10) questions
5.Agreement
a. Used when traditional estimates such as stability, equivalence, equivalence and stability, or internal consistency are not applicable
b. Typically some form of agreement is used (e.g., raters agreeing with one another)
c. Situations in which this estimate is used
1.Observational measures - agreement between raters making the same observation
2.Insufficient numbers of test items on an instrument - agreement across the percentage of responses that are the same for several subjects
3.Data with highly skewed distributions - percentage of agreement in the number of subjects |
|
|
Term
| Identify the conditions affecting reliability |
|
Definition
1.Length of the test (i.e., longer tests are typically more reliable)
2.Subjects
a.Instruments used with heterogeneous samples typically have higher reliability than those used with homogeneous samples
b.Scores for older subjects are typically more reliable than those for younger children
3.Trait being measured (i.e., cognitive traits are more reliable than affective characteristics) |
|
|
Term
| identify the ways by which reliability can be enhanced |
|
Definition
1.Standardized administration procedures (e.g., directions, conditions, etc.)
2.Appropriate reading level
3.Reasonable length of the testing period
4.Counterbalancing the order of testing if several tests are being given |
|
|
Term
| Explain the relationship between validity and reliability |
|
Definition
| reliability refers to the tool of measurement, validity refers not only to reliability of the measure, but to how the results are evaluated and how conclusions are drawn from the measurement; how well the study studies what it purports to study |
|
|
Term
| Explain the difference between cognitive and non-cognitive measures |
|
Definition
Cognitive measures focus on what a person knows or is able to do mentally
Non-cognitive measures focus on affective traits or characteristics (e.g, personality traits, attitudes, values, interests, preferences, etc.) |
|
|
Term
| explain the difference between commercial and locally developed measures |
|
Definition
Commercially prepared measures are developed for widespread use with a focus on technical merit
Locally prepared measures are developed by a researcher for specific situations with some, but not extensive, concern for technical characteristics |
|
|
Term
|
Definition
| the most frequently occurring score |
|
|
Term
|
Definition
| the score above and below which one-half of the scores occur |
|
|
Term
|
Definition
| The arithmetic average of all scores |
|
|
