The Alignment Research Center has published a description of its current technical research agenda, centred on building structural descriptions of neural networks that can estimate the probability of catastrophic outputs more efficiently than random sampling.
The post, attributed to ARC, explains that the organisation has reoriented its research since 2025 around a more specific goal: outperforming random sampling when estimating properties of neural network outputs. The claimed advantage of this framing over previous ARC goals is that it is more concrete and more directly tied to applications in AI safety.
The safety motivation, as set out in the post, concerns a class of AI alignment approaches that involve adversarial training: building a catastrophe detector that classifies model outputs as catastrophic or not, then training the model to minimise the probability of outputs the detector flags. The post identifies two obstacles. First, if the model is more capable than the catastrophe detector, it can find outputs that fool the detector without triggering a flag. Second, even with a perfect catastrophe detector, computing the expected frequency of catastrophic outputs is impractical if catastrophes are rare — because the most direct approach, drawing random samples, is extremely slow when the target event almost never occurs.
ARC’s proposed response to the second obstacle is to use structural understanding of the model to estimate rare-event probabilities far more efficiently than sampling. The post gives a worked example: if a catastrophe detector can be understood as a conjunction of three independent predicates, and each predicate is satisfied with probability one in a million, sampling would require approximately 10^18 trials to estimate their joint probability. A mechanistic understanding of the predicates’ independence, combined with sampling each predicate separately at roughly 3 × 10^6 samples each, reaches the same estimate orders of magnitude more efficiently.
The post introduces what ARC calls the matching sampling principle (MSP) — described as a semi-formalisation of the belief underpinning the research agenda. The principle, stated at a high level, is the conjecture that there exists a mechanistic estimation procedure which, given sufficient advice about a model’s structure, performs at least as well as random sampling in mean squared error for any given computational budget. The post notes that the full formal statement of the MSP is more complex, and walks through several versions and the reasons earlier formulations were revised.
ARC distinguishes its goal from most mechanistic interpretability research, which the post says is aimed at partial, human-legible understanding of neural networks. ARC’s target is instead full, algorithmic understanding — structural explanations that may be as large and as incomprehensible to humans as the model itself, but that can be used by an algorithm to estimate the model’s outputs. The post states: “Our goal is not to have a human look at the structure and estimate the expectation of C(M(x)). Instead, the goal is to invent an algorithm that takes as input the explanation and estimates expectation of C(M(x)) based on that explanation.”
The post reports that ARC has made progress toward matching sampling in specific contexts, including random MLPs and trained two-layer MLPs, though it describes these as preliminary. ARC also notes it is hiring researchers to work on this agenda.
The AlgZoo model collection — published separately by ARC on the same date — provides small benchmark models intended to test whether mechanistic estimation approaches of the kind described in this post can be made to work on networks of increasing complexity.