School district administrators and principals are inundated with salesmen peddling computers and software programs. Many claim that scientific research proves their wares work. Can they be believed? The researchers at the Abdul Latif Jameel Poverty Action Lab (J-PAL), an organization inside the economics department of the Massachusetts Institute of Technology, scoured academic journals, the internet and evaluation databases and found only 113 studies on using technology in schools that were scientifically rigorous.
“Education technology is an area where innovation has outpaced rigorous research,” said Vincent Quan, who runs the North American education unit at J-PAL. “We wanted to find all the studies and distill the main lessons so that decision makers can decide which programs to scale up and invest in.”
To meet J-PAL’s high standards, the study either had to be a randomized controlled trial, in which students were randomly selected to try a technology and studied alongside students who didn’t try it, or it had to be a “regression discontinuity design,” in which students with statistically similar test scores were studied, but those just below a test-score threshold tried the technology, for example, and those just above it didn’t. Both types of studies are expensive and typically take two years or more to conduct — time and money that ed-tech entrepreneurs usually don’t have. Technology can become obsolete by the time the results come out.