
The Road to Better Assessments
Best PracticesOver the past couple of weeks, there have been some high-profile cases of institutions that rolled out a remote assessment, to less than ideal results. Rather than let it get swept under as last week’s news, we think this presents a unique opportunity to talk about what could have been done differently, using some of the strategies which, in our experience, result in more successful implementations.
We’ll focus on one case from last week, the National Autonomous University of Mexico (UNAM). A quick recap, to bring you up to speed:
What Happened at UNAM
UNAM is a prestigious Mexican university. The institution admits around 50,000 students each year, about half of whom feed directly from their own prep schools, the rest via the entrance exam in question. Typically, that means students of lesser financial means are taking the test. In an effort to reach more students, UNAM administered a remote, digital exam using two proctoring services (neither was Proctorio). The results were not good: 5.5% of students scored a 110 (maximum score is 120), up from 0.9% a year ago.
Everyone seemed surprised by this outcome. To remediate, UNAM is asking 58,000 students who have some recorded suspicion of cheating to re-sit the exam. Protests and much public outcry have resulted.
Our conclusion? This situation was both predictable and at least partially avoidable, and there is a silver lining: specific strategies could have prevented this outcome, and we can all use this situation to learn how to implement better. Below are four non-technology-based strategies, followed by three proctoring-technology-specific recommendations, for our readers to consider and adopt where possible.
Non-Tech Recommendations
Avoid all-or-nothing events. Research into what motivates students to cheat will tell you over and over that you should avoid all-or-nothing events when possible. They incentivize otherwise honest people to bend the rules, since all of the risk/reward is riding on one outcome. This is true of any exam, remote or in person. James Lang’s book Cheating Lessons or, for more recent research, Rettinger and Bertram-Gallant’s The Opposite of Cheating, dive into the importance of creating the right environment, and that approach continues to bear fruit for those who employ it. UNAM’s environment, by contrast, did not do this. It was an entrance exam, yes, but other data points could have been considered with more weight (accepting multiple types of exams, evaluating essay submissions, etc.) to lessen the pressure riding on this one event.
Lower the stakes. As discussed in the summary, the examinees in this case were applicants from lower socioeconomic situations, which means that attending, or not attending, a prestigious university may be the difference between a quality life for your entire family and a comparably miserable one. Given what is at stake, it’s not surprising that many honest people are going to try to tip the scales. The solution? Provide alternative entry points, and emphasize them. In the United States, many states have created a community college pipeline that serves this purpose, letting students who might have failed the entrance exam route through an affordable community college and matriculate in their junior year.
Modernize assessments, using the available research. The exam itself used by UNAM was reported as being multiple choice. This design has been shown to be abnormally vulnerable to cheating technologies (it’s easy for a bot to fill in bubbles online, much harder for it to write essays or record a video response). Multiple choice assessment, even before AI technology, was a problematic model, since you can guess the right answer, and many students simply learn to strategize choosing the most likely answer rather than actually learning the material. There is a trade-off: multiple choice is certainly easier to grade, but it works more effectively in formative, low-to-mid-stakes situations.
Consider the larger context. When people think it’s easy to cheat, they are more likely to do so, if only because they assume everyone else will be doing so. Students read constantly about others using AI to cheat, and probably assume that everyone else will attempt it. This increases the pressure to use it themselves. An additional, fascinating area of research is the question of whether students would even consider it cheating, given how the subject is often presented in online forums, with products with names like “homework help” that imply, “hey, we are here to help you get a good grade!” Discussions from the assessment provider on what is expected, what constitutes cheating, and well-reasoned ethical guidelines make a measurable difference for the average test-taker. Include them.
Where Technology Fits In
If you are wondering whether the proctoring companies involved should have resolved all of this, the answer is not entirely. No technology can operate in isolation, and asking it to do so is like putting a medieval sentry alone at the gate. The bulk of cheating deterrence can and should happen before students even get to the exam, and it’s possible to do so. Technology plays a part, which we will discuss next, but it needs the right context to maximize its effectiveness.
One last thought on where technology is not the solution. The main data point referenced in the articles showing an increase in cheating is the rate of very high test scores going from 0.9% to 5.5%. To our thinking, that points to AI specifically being used (most bots won’t intentionally get an answer wrong), but it doesn’t prove that cheating necessarily increased measurably. If a lower score still gets you in, and you had to sit the exam in person, you may have still cheated and cleared the bar; the AI just cheats better than a human can. Your proctoring technology may simply be recording incidences of cheating that weren’t being caught in the classroom or testing center before. Of course, there may be more data that the university is looking at. From what is reported, it is not necessarily a given that cheating increased so dramatically, just that it was caught.
High-Tech Recommendations
On the technology front, there are indeed useful lessons here as well. UNAM employed a combination of tools, Respondus LockDown Browser for device control, and a company called Terrarium for what sounds like the monitoring portion. Here’s what we think they could have done differently, and what others can learn from it.
Use one expert solution. Combining two solutions is much more likely to leave gaps. This is not the place to divide the responsibility. As anyone who played outfield can tell you, you have a better chance of catching the ball when one person takes responsibility. In this case, neither company is likely to be sufficiently expert in what the other’s capabilities are, and that means gaps, even if both are expert in the comprehensive version of their own solution.
Invite your vendors in and lean on their expertise. It’s possible UNAM did this, we don’t know. But for an exam of this scale, you have the right to lean on your vendor’s expertise, and it might even be prudent to have them there on the ground with you. A good vendor should be able to tell you what their technology does and does not do. Be careful of those who claim something like “we can stop all AI.” Instead, know where the technology being used ends, and plan around those gaps; your vendor should be able to help you identify them.
Practice small first. Don’t let your first swing be a high-stakes, large-scale exam. A good approach is to start small and gradually build to the larger, more important scenario. If a large exam is the only kind you run, start with small cohorts until you’ve worked out any gaps and are confident in your results. Most of the problems UNAM experienced would, presumably, have surfaced in a limited, 20-person Biology 101 exam, for example, or by rolling out remote testing to only 1,000 applicants in the first year.
The Bottom Line
The great news is that the situation UNAM faced can be avoided, and our hope in publishing this article is to promote good remote testing hygiene, industry-wide. There is real data and research showing that large-scale remote testing is successful when implemented properly. When we work together and rely on experience and the knowledge that is out there, it expands access and opportunity to those who need it most.
Further Reading
For the non-tech solutions, we’re indebted to the academic writers below in particular, and would recommend their work to anyone in this space. In addition, one of the best centralized sources for the strategies discussed here is the International Center for Academic Integrity: www.academicintegrity.org
Lang, James M. Cheating Lessons. President and Fellows of Harvard College, 2013.
Rettinger, David A., and Tricia Bertram Gallant. The Opposite of Cheating. University of Oklahoma Press, 2025.
