Peer Review Is Failing Us and What That Means for the Future Past of AI

At the time of writing the initial draft of this post, I was in the weird limbo state of having to wait for the reviews of my thesis. It’s been a number of months, which is the standard time allotted to these kinds of things. Still, it’s an uncomfortable limbo state to be in. Furthermore, I was in the midst of reviewing for ICML 2025, which has been a less than stellar experience (low quality submissions, low quality fellow reviews, terrible review format). Distilling these feelings led me to conclude that as an academic, your career and your future, in many ways, is in the hands of strangers, much more so than in other jobs, where your career tends to be made by the decisions of your immediate superiors and those of the C-suite (layoffs et al.). Of course, when you deliver software, you still need to appeal to customers, please your supervisors and all that. Yet, your entire value to all hiring managers in your field is not being decided by a stranger’s anonymous comments on the summary of a year’s worth of work.

If you’re in academia, particularly in CS/AI, then none of the points I am going to make will be novel to you, although for those coming from other fields, it could be that publishing in journals is the mode-du-jour. Still, for people outside of academia, this might be a (one-sided, opinionated) look at a part of the process of science that non-academic outsiders aren’t all that familiar with. It’s also a way to help me put the disillusionment I have built over the years in academia into writing. As a young, bright-eyed master’s graduate, graduating into a pandemic, I was convinced that dedicating my future to the creation of knowledge and pushing the frontier of science further, in which little way I could was my calling. However, it turns out that what you want to participate in as a student is Science 1 (research, novel ideas), and the reality is Science 2 (politics, marketing, bullshitting, grants, soft fraud).¹

Of course, that is not to say that your research ultimately doesn’t matter. It does. Yet, the feeling I’ve gotten from peeking behind the curtain is that your research only takes you so far. Once you cross a certain minimum quality requirement, the crux of getting into venues that secure future positions is mainly that of selling your ideas, citing the right people and using the right jargon to please the reviewers, and to make them feel smart (or stupid, really depends on the venue). My view is that the degree to which research is judged on its scientific merits is staggeringly low, compared to the value of superficial in-group signifiers and presentation.

This also means that unless your advisor knows the shibboleths of the top-tier conferences, your research may be be great, and it still you won’t ever make a career. This divide seems particularly egregious when you consider the relative inaffordability of attending those conferences when your salary comes out of a developing country’s university budget.²

So What Exactly is Peer Review and What’s the Problem?

Once your paper is written and submitted, it will be reviewed by fellow researchers who sometimes may have some idea, or, in rare cases, might have even heard of the topic of your research before. This process is called peer review and it sort of works some of the time. It’s also the “best thing we currently have”. It is also a system of uncompensated labor, which publishers and conference organisers seem happily entitled to.³

While reviewing for a conference will usually get your name on a list, so you can point to it in your CV (wow!), it’s not really work that feels (or is) rewarding. The rebuttal phase⁴ can be stressful, with authors not always taking valid criticism too well and papers that are strong candidates being overlooked by the ACs at the end of the day.⁵ You’re still supposed to do it, as “a duty to the community”, just like you’re supposed to floss or abdicate your sins. If you know the system, it comes as no surprise that the work is unpopular, and both conferences and journals struggle to find reviewers year in year out, especially as conferences grow.⁶

The enduring shortage of reviewers has led some conferences to institute policies (or rather “schemes”) where the authors of a submitted paper have to nominate one of their own to review a number of other submissions. For the observant reader, it should be rather clear, how this sets up a system of perverse incentives, as you are now reviewing other papers that are vying for the same pool of limited spots in the conference as your own submission. In my experience, this system leads to reviews giving vague dismissals that are hard to argue on a factual level, making it ever harder to evaluate papers on their merits for the ACs. The reviews also tend to be of very low quality, offering no signal for informed decision-making. Due to the high popularity of this system, the organisers of some conferences now resort to threats and penalties to have authors comply with their demands. On top of that, there seems to be a race to the bottom as far as the reviewing formats go, with an ever growing list of boxes to tick and increasingly specific text boxes adding to both the popularity of the reviewing process as well as to the quality.

To throw another newt into the bitter stew arising from old systems not keeping up with a rapidly growing field, the general unpopularity of reviewing and the increasing gatekeeping of jobs behind publications in certain venues: the system fundamentally is not designed to deal with bad faith actors. As careers are now largely made by a series of metrics thought up by bureaucrats, this leads to various forms of fraudulent behavior. On the low end of the scale, reviewers will ask authors to include citations to their own work, no matter how (ir)relevant to the paper it might be. Cooking the numbers is the name of the game if you want to make it in Science 2. In more serious cases, coordinated groups undermine the process ensuring that their papers get into the venues through gaming of the bidding processes.

All this being said, I don’t think getting rid of reviews altogether would be of any help to the problems, and I unfortunately don’t have a great solution either. A functional system of review could be established by professionalizing the work, however this will likely lead to the establishment of a class of professional gate-keepers, and there is a good chance that it might incentivize corruption and a lot of moaning about bureaucracy. However, I think that establishing such a system would at least give the reviewers the time and resources to deal with bad-faith behavior and allow the scientific community to establish a set of transparent standards on which papers can be judged, rather than the group that is also trying to get their paper into the same venue barely reading your paper.⁷

The Future Past of AI research

I sincerely believe that in 30 years, we will be looking back at this era of AI research in ways that are similar to how we look at all the discredited psychology research done in the 1970s these days. The systemic failures in the reviewing process, the incentives of academia rewarding the gaming of statistics, and the fact that large parts of the research at conferences is what amounts to non-reproducible PR-fluff for various tech companies, all add together to something that appears to easily overwhelm the little resilience that was built into the system.

This combines with a lack of rigor and proper statistical analysis that encourages the uncritical publication (and overselling) of any type of number-goes-up style result for many of the hundreds of ill-conceived benchmarks that have flooded into the field. Do we care whether a given result could be largely noise or a lucky run or data leakage? Well, we have made a meaningless ablation study to placate the reviewers coming from an entirely different (sub)field and so they would check all the 25 boxes on the review form for good scientific conduct. Fine, no objections, ship the paper, publish the model, earn some investor money, science marches on. Where to? I don’t really know, but I am still hopeful that things can get better after the hype is over, the bubble has popped and we are all waking up a little more sober than before.