#### Correlation

• estimate portion of variance that is error variance
• degree of consistency or agreement between two independently derived sets of scores
• stated as a correlation coefficient -1.0 to +1.0
• Example

#### Pearson's Product-Moment Correlation Coefficient

• person's position in group and amount of deviation from group mean
• significance depends on size of sample
• 10 cases r=.40 not significant
• 100 cases r=.40 significant
• more detail

#### Test-retest Reliability

• repeat identical test on a second occassion
• correlation between scores obtained by same person
• The scores should be the same minus the error variance
• error variance corresponds to random fluctuations in performance
• i.e., broken pencil, illness, fatigue...
• practice effects
• must state interval, as r decreases with time
• >6 months = Coefficient of Stability
• MMPI example

#### Alternate-Form Reliability:

• to avoid problems with test-retest
• use of comparable forms
• measures "temporal stability"
• also measures consistency of response to different item samples
• concept of "item sampling"
lucky break versus hard test... what extent to scores on the test depend on factors specific to selection of items
• short interval = measure of relationship between forms
• long interval = measure of test-retest and alternate forms
• very time consuming and work intensive
• more detail

#### Split-Half Reliability

• single administration of test - split in half
• two scores for each person
• measure of consistency of content sampling
• odd versus even
• more detail

#### Methods of Estimating Internal Consistency

Method used to determine the extent to which all the items on a given test are measuring the same skill (e.g., the test is "consistently" measuring the same skill.)

• Spearman-Brown Formula.
• Inter-item Consistency Formula:
• The degree of correlation between all the items on a scale (e.g., Cronbach's alpha).
• Tests of Homogeneity:
• The degree to which a test measures a single factor, or the extent to which items in a scale are unifactorial.
• Tests of Heterogeneity: The degree to which a test measures different factors.
• The Kuder-Richardson formulas: Used on tests with dichotomous items.
• Kuder-Richardson 20 (KR20) - mean of all possible split-half coefficients
• The Coefficient Alpha: used for tests that contain non-dichotomous items (items that can individually be scored along a range of values, such as attitude polls and essay tests).
• Most often used for survey scales and objective assessments
• yields a lower bound estimate of reliability
• can be negative if inter-item correlations are negative
• if items are dichotomously scored, coefficient alpha equals KR20 value

#### Scorer/Inter-Rater Reliability

• measure of examiner variance
• objective versus subjective measures
• high degree of judgement = high chance of variance
• measure degree of consistency between two or three examiners
• .80 or better is good
• can use Pearson's, Spearmen's or Kappa
• more detail