Assessments: Paper vs Online

For my reading curriculum, students have to take a weekly test that gauges their comprehension. Recently, I have given the students the option of taking the assessments online. Currently my two classes are split in the way they take the assessments. Some take it on paper, using their personal iPad/tablet or using a laptop from the computer cart.

Has anyone ever used an online component to give an assessment? I noticed that some students who struggled are doing better and some students who did well had lower scores.

Thanks for the feedback!

Nicole C