EPSY 551: Item Response Theory / Rasch Measurement

4 Credit Hours. CRN: 43708.

Semester: Fall 2023
Professor: George Karabatsos
(home page)
Classroom:
Online asynchronous (See note below).
E-mail: georgek@uic.edu
Course Material:
In BlackBoard.

Computer Lab: ETMSW2027.
Office Hours:
Monday 9am-12pm (EPASW 1034)
,
or by appointment.
Please email me to schedule a meeting during office hours.

Classroom: Online asynchronous. A new video lecture is presented and recorded through Zoom (via BlackBoard)
every Monday 2-5pm during the semester. Students may either attend these lectures live, while asking any questions; or otherwise, may watch the class (lecture) recordings 
which will be made available in BlackBoard by Tuesday 6pm. Questions may be asked by e-mail and/or during scheduled office hour meetings.


Course Description
:
Item Response Theory (IRT) models continue to see vastly many applications in the educational, psychological, health, and other social sciences. This course teaches how to apply classical and contemporary IRT models for the psychometric analysis of data arising from examinees' responses to items of a test (e.g., examination, rating scale questionnaire), and for the analysis of judge ratings of examinee performance on test items. In an IRT model analysis of item response data, the main objective is to estimate each examinee's ability of the test, each item's difficulty level, and perhaps, estimate the regression coefficients of one or more (examinee-level, item-level, and/or judge/rater level) predictor variables. IRT models that are covered by this course include Rasch models, the 2-parameter logistic model, the 3-parameter logistic model, the nonparametric monotone homogeneity model (via kernel regression), graded response models, the generalized partial credit model, Rasch models with predictor variables, as well as finite-mixture and infinite-mixture (e.g., Dirichlet process mixture) versions of these IRT models. This course also covers factor analysis. The course does not assign special preference to certain IRT or Rasch models, but instead allows students to exercise their respective individual preferences.
While this course focuses and grades on applications of various IRT models, this course also covers the theoretical foundations of IRT models, and the modern methods used to estimate the parameters of IRT models from data (including point-estimation and Bayes posterior estimation). These models will be illustrated through the analysis of data using various software packages including  R packages such as mirteRmmixRasch, lavaan, fungiblelme4, as well as IRTPRO, PARSCALE, HLM, Bayesian Regression, SPSS, MATLAB, and other software packages that will be made available to students mostly free-of-charge. (SPSS is available for purchase through the University of Illinois Webstore).
Assignment (exam) problems will be practiced and worked on in each class lecture/session.

Prerequisites: At least two graduate courses on quantitative methods.

Books and suggested readings. (The book below, in bold, is available in the UIC bookstore):
De Boeck, P., and Wilson, M. (2004). Explanatory Item Response Models. Boca Raton, FL: Chapman and Hall/CRC.
Embretson, S.E., & Riese, S.P. (2000). Item Response Theory for Psychologists. Lawrence Erlbaum Associates, Mahwah, NJ.
Karabatsos, G., Walker, S.G. (2009). A Bayesian Nonparametric Approach to Test Equating. Psychometrika, 74, 211-232.
Kline, P. (1994). An Easy Guide to Factor Analysis. NewYork: Routledge.
Levy, R., and Mislevy, R.J. (2016). Bayesian Psychometric Modeling. Boca Raton, FL: Chapman and Hall/CRC.
Meijer, R R., & Nering, M.L. (1999). Computerized adaptive testing: Overview and introduction. Applied Psychological Measurement, 23, 187-194.
van der Linden, W.J. (2016). Handbook of Item Response Theory, Volume One: Models. Boca Raton, FL: Chapman and Hall/CRC.
van der Linden, W.J. (2016). Handbook of Item Response Theory, Volume Two: Statistical Tools. Boca Raton, FL: Chapman and Hall/CRC.
van der Linden, W.J. (2018). Handbook of Item Response Theory, Volume Three: Applications. Boca Raton, FL: Chapman and Hall/CRC.

Important Dates:
August 21, M, Instruction begins.
September 4. Labor Day holiday. (Class WILL be given on this day).
November 23–24, Th–F, Thanksgiving holiday. No classes.
December 1, F Instruction ends.
December 4-8, Final Exam Week: Final Paper due by Wednesday 11:59pm.

COURSE SCHEDULE
Week Topic
1

What is an IRT model? IRT foundations
Basic data structure in IRT modeling.
The general unidimensional IRT model.
The item response function (IRF) (item characteristic curve,
item-step response function (ISRF), item category-response function.

2

IRT foundations (continued)
-- The three properties of all psychometric models
(unidimensionality, local independence, monotonicity of the IRF/ISRF).
-- Invariant item ordering.
-- Examples of IRT models under this general framework.

IRT/Rasch models for data analysis
Parameter maximum likelihood estimation methods for IRT models.
Model selection methods
3
IRT/Rasch models for dichotomous item scores
Rasch, 2-parameter, and 3-parameter logistic models.
Illustrative applications of models on real data.
4
IRT/Rasch methods for polytomous item scores
Real data illustrations of various IRT models for polytomous item scores.
Graded response models, generalized partial credit model, Rasch rating scale model.
5
Nonparametric IRT: Kernel regression approach
Kernel estimation of the IRF, the ISRF, and the category response function.
Investigating measurement unidimensionality (investigating IRF/ISRF monotonicity).
Estimating examinee ability, and item difficulty.
Illustrative applications of models on real data.
EXAM #1 is due on Friday this week in Blackboard
(Exam on dichotomous parametric IRT models).
6
Factor Analysis
Exploratory Factor analysis including Exploratory Bifactor Analysis.
Confirmatory factor analysis.
Illustrative applications of models on real data.
7
IRT/Rasch modeling with covariates via Hierarchical Linear Models (HLM)
Rasch model as a Hierarchical Linear Model.
(Rasch) analysis of test items, rating scales, and judge ratings.
Investigating Item Bias (Differential Item Functioning).
Comparing test performance across different groups of respondents.
Incorporating additional predictor variables in psychometric analysis.
Illustrative applications of models on real data.

8

IRT/Rasch modeling with predictor variables via HLM and Facets
Illustrative applications of models on real data.
EXAM #2 is due on Friday this week in Blackboard
(Exam on polytomous, kernel nonparametric, or hierarchical IRT models).

9

Finite Mixture (latent class) IRT models
Examples of models.
Illustrative applications of models on real data.
10
Bayesian nonparametric (infinite-mixture) Rasch models for data analysis
Bayesian estimation of the posterior distribution.
Various applications of Bayesian semiparametric mixed Rasch models.
Analysis of test items, rating scales, and judge ratings.
Investigating Item Bias (Differential Item Functioning).
Comparing test performance across different groups of respondents.
Incorporating additional predictor variables in psychometric analysis.
11
Bayesian nonparametric mixed Rasch models for data analysis
Illustrative applications of models on real data.
12
Equating Test Scores and test items
The equipercentile approach to test equating (with bootstrap confidence intervals).
Linear equating. Rasch item equating. Bayesian nonparametric equating.
13
Computer Adaptive Testing using IRT
EXAM #3 is due on Friday this week in Blackboard
(Exam on mixture IRT models).
14
Cognitive Diagnosis Modeling
15
Student Presentations
16,
Exam
Week
FINAL PAPER DUE on Wednesday (December 6) of Exam Week. Please submit final paper in Blackboard.

*The instructor reserves the right to make any changes in the course he determines academically advisable.
Changes will be announced in class. It is your responsibility to keep up with any changed policies.


Course Assignments
Exams 1, 2, and 3: Each take-home exam gives you the opportunity to practice with applying and explaining various IRT/Rasch models through analysis of real data sets.
Short paper and presentation: An 8-page (double-spaced) paper, on applying and explaining various IRT/Rasch models, to be presented in class near the end of the semester.
For each exam, and for the the paper, please show all work and relevant output, and place all raw output in the Appendix.
Note
: You have the option to not do an in-class presentation, in which case you would be required to submit a 12-page (double-spaced) paper.
That being said, historically, one of the more enjoyable aspects of the course has been watching and giving the in-class presentations on IRT models.
There is no penalty in grade for choosing not to do the class presentation option.

Grading Policy
: Exams 1, 2, 3, and the final paper (with optional presentation) are each worth 22.5% of the final grade (together, they are worth 90%).
Class participation is worth the remaining 10% of the final grade. Class participation means submitting all of your course assignments on time according to their respective due dates.
Also, I am happy to receive questions from students about IRT during my live class lectures.
Final grades will be given out according to the following scale:

A
90% - 100%
B
79% - 89%
C
68% - 78%
D
57% - 67%
F
56% - Lower

The amount of student class participation will be used to decide borderline grades.
Students will spend substantial amounts of time reading, and on the computer.
It is assumed that students will exert individual initiative in solving computing/analysis problems as they arise.
There are no exceptions to the above grading scale, and no extra credit work will be accepted.
Incomplete grades will be considered for students with extenuating circumstances.
Poor performance on assignments will not be considered in a request for an incomplete.
All assignments submitted late to BlackBoard, past the stated assignment due date,
will receive a grade of zero. No exceptions will be made to this policy.

*The instructor reserves the right to make any changes in the course he determines academically advisable.
Changes will be announced in class. It is your responsibility to keep up with any changed policies.

DATA ANALYSIS PAPER (AND OPTIONAL PAPER PRESENTATION):
The data analyses will consist of the relevant output from the software programs and a complete report stating the results.
You may supply your own data or you may solicit faculty (education or other) for data.
Some data sources: Open Psychometrics, PIRLS, TIMSS, PISA.
If you are doing a presentation, the final paper should be at least 7 double-spaced pages (not including Appendix),
using 1-inch margins and APA or other standard scientific style.
(Otherwise, if you are not doing a presentation, you need to do a 12 page paper; see above).
Please discuss only the relevant results of your analysis, within the main body of the paper.
Please put all data analysis output in the Appendix of your paper.
The optional presentation has a limit of 20 minutes (about 15 PowerPoint slides).
If you will do a presentation, please hand me a hard-copy of your PowerPoint presentation on the day of your presentation.


The paper (and optinal presentation) must at least include the following sections:

Introduction
Describe in detail the substantive problems you will be addressing in this research study (5 points).

Methods

Describe sample characteristics (3 points).
Describe the items on your test(s) (including their number and scoring format) (3 points).
Describe the construct you intend to measure with the test(s) (3 points).
For data analysis, use one or more IRT models for data analysis.
Fully describe your IRT model(s) and the parameter estimation methods (15 points)
If you intend to equate test scores, fully describe the equating methods you will implement.
If you will implement one or more IRT models and plan to compare them, describe the model selection method you will use.

Results
Summarize the results of your analysis, including estimates of IRT model parameters (10 points).
If necessary, justify any modifications you make to your test (e.g., removing items).
Please discuss only the relevant results of your analysis. (15 points)

Discussion
What modifications (if any) would improve the test? (3 points)
What are the implications of your study, with respect to the measurement and applications in the field of interest? (3 points)

Please provide appropriate handouts and develop meaningful overheads for your presentation.

Disability Services:
UIC strives to ensure the accessibility of programs, classes, and services to students with disabilities.
Reasonable accommodations can be arranged for students with various types of disabilities,
such as documented learning disabilities, vision, or hearing impairments, and emotional or physical disabilities.
If you need accommodations for this class, please let your instructor know your needs and s/he
will help you obtain the assistance you need in conjunction with the Office of Disability Services (1190 SSB, 413-2183).