No. 034
PaperRaven, LLM peer review framing, AlphaGenome 2
A common design question appears across several items this week: at which point in the research pipeline should AI output be checked by an independent process? PaperRaven addresses it at the literature stage by refusing any citation the model proposes but cannot locate in OpenAlex or Crossref. The DIME team's code-review agent addresses it at the analysis stage, prompted by an audit finding coding errors in 26 of 110 published economics papers despite most being computationally reproducible. A NeurIPS benchmark addresses it at the review stage and finds that model selection alone produces a 50-percentage-point gap in error detection rates. The LLM framing experiment shows what is at stake when that check is absent: a single prompt change raised aggregate acceptance rates more than fourfold on the same set of published papers.
Literature and Data Infrastructure
-
Aaron Clauset, November 12 2026
The design separates generation from verification: the model proposes, code checks each DOI against OpenAlex and Crossref and refuses anything it cannot confirm, and every inclusion or exclusion decision is logged with its reasoning, so the output is a bibliography paired with a full audit trail.
-
AlphaGenome 2: deciphering how cells read DNA to better understand disease
Google DeepMind, November 11 2026
Researchers can query through a non-commercial API or download precomputed scores for 7 million common variants and a precomputed atlas of enhancer-gene links, and on the TenK10K benchmark the model found 30 percent of causal immune-cell variants versus 16 percent for its predecessor.
Code Quality and Review
-
AI is writing more research code. Who is reviewing it?
World Bank Development Impact (Ankriti Singh, MarĂa Reyes Retana Torre, Maria Jones), November 3 2026
In an audit of 110 published economics and political science articles, 26 contained substantive coding errors despite 85 percent being computationally reproducible, and the team has released a public agent skill of more than 60 checks that runs a read-only code review in two phases and reports findings by severity.
-
Auditing AI peer reviewers: a benchmark on real scientific papers
arXiv (Zubeldia, Bolliet, Villaescusa-Navarro, Villanueva-Domingo, Cranmer), November 12 2026
Using 357 deliberate errors of four types injected into real astrophysics and machine learning papers, the benchmark found that model choice has the largest effect on detection rate: across the same papers and prompts, three frontier LLMs differed by roughly 50 percentage points in total detection.
Peer Review and Verification
-
Down the Rabbit Hole of LLM Peer Review
Radiology: Artificial Intelligence (Bastien Le Guellec, Su Hwan Kim), November 11 2026
A single sentence telling the model a manuscript had already been revised once raised aggregate acceptance rates from 4 to 18 percent across four models on 200 published papers, and Gemini 3.7 Flash's rate from 15.5 to 68.5 percent, leading the authors to argue that a reviewer must be accountable for the experiments it requests and that an LLM cannot meet that standard.
-
AI's ability to 'autoformalize' proofs is changing math
Quanta Magazine (Konstantin Kakaes), November 11 2026
Anthropic's agents formalized Fermat's Last Theorem in 30 hours, compressing what required 11 days in September, and formalization expert Kevin Buzzard now expects formalization of new results on the fly, shifting the question of mathematical correctness from peer review to machine-checkable verification.