CAT: Computer Adaptive Testing

On July 15th, the Superintendent of the state of Virginia released this memo stating that the CAT (Computer Adaptive Testing) format of the Math 4 and 5 SOL test would begin to be administered in the Spring of 2017.  A second memo, found here, also stated that students in grades 3-5 would also begin the CAT test in Reading this Spring.  So what does this mean for you and your students?

CAT testing has actually been around for quite some time.  It has been used in the Algebra Readiness Diagnostic Test since 2002.  Students taking the 6th grade math SOL have been taking the CAT test since 2014, and 3rd graders took the math CAT test for the first time this year.

Here's a few more things you should know:

The testing experience is shorter, according to the VDOE.
Not only does the test contain fewer items, in my experience the testing "time" is much shorter.  Because students can no longer go back and check their work, they simply are not testing as long.

The goal of CAT testing is for students to be assessed on their readiness level.
Students start each test with a "moderate" level question.  Answers are scored as each question in answered. Based on the student response, the test then selects another question from a question bank.  While the level of question difficulty may change, students are only assessed on their particular grade level.  For example, Math 5 students will only receive 5th grade questions.  Because each student's testing experience is individualized, CAT testing inherently improves testing security.

Students can no longer "flag for review" or go back to check their work.
Because students need to select an answer for the computer to know what type of question to give them next, students must select an answer choice to move on.  On the Math tests, students cannot flag for review, and there is no longer a "Previous" button.

At SOL committees this week, it was mentioned by the VDOE's Assessment Team that this will look a little different on the Reading CAT tests.  Students may be able to go back to questions for one passage set and "bookmark" questions, however once the passage has been completed they will not be permitted to return.  Students will get a "warning" message once they've finished a passage, reminding them that they cannot got back. (Thanks to Melissa Dalton for clarification on this!)  Because this is the first time the VDOE is doing a Reading CAT test, I'll be interested to see how they roll out "official" information on this!

It is also important to know that on TEI questions, as soon as one selection is made, the "Next" button will appear.  This does not mean that the student has selected all of the appropriate answers.

Another change is that the "Section Review" screen has been eliminated, because students will have had to answer each question in order to proceed.  Finally, there is no "Final Submit" button.

Scoring reports are different.
The scoring scale remains the same (0-600); however there is no "cut" score. According to the VDOE, students are scored both on answering questions correctly, as well as the question's level of difficulty.  Because each student's test is different, each student receives an individual score report that includes their score and item level performance information.

There are some similarities to traditional testing.
CAT tests still contain multiple choice questions and TEI questions.  Students will also still receive a Student Authorization to Test ticket, and the general instructions are similar in nature.  All of the testing tools remain the same.

It's also important to know that CAT tests still allow for testing accommodations.  However, Read Aloud accommodations will have to be one-on-one testing because each test will be different from one another.  The examiner's manual goes further into depth about the changes to testing accommodations.  Be sure to read this section carefully!

I highly encourage everyone to check out the VDOE's resources on CAT testing, including the CAT training tests for students.  The SOL testing blueprints for English and the SOL testing blueprints for Math have been updated to reflect the number of operational and field test items for each CAT test.  For example, the traditional Math 3 SOL test had 50 items; the new CAT test only has 32!

Overall, I believe the changes are heading in the right direction for our students!  What are your thoughts?



Pin For Later!