Two base rates, two weights: base-rate neglect has a second axis

This paper, Two base rates, two weights: base-rate neglect has a second axis by A. Y. Shavit, argues that base-rate neglect is not one mistake but two separate errors. The central claim is that "Turning the co-occurrences a learner actually experiences into a usable judgment — the step Bayes' rule performs — means correcting for two base rates, not one. The first is the familiar prior: how common the outcome is. The second is the cue's own base rate — how often the cue occurs at all — and correcting for it is a genuinely separate step."

The paper formalizes this with a two-weight Bayesian equation: "ρpr is how much of the prior correction the learner applies; ρev is how much of the cue correction. A weight of 1 applies that correction in full (calibrated); a weight of 0 skips it. ρev < 1 is the cue-density effect, under-dividing by how common the cue is. ρpr < 1 is classical base-rate neglect, under-using how common the outcome is."

The key insight is that A learner can make either correction, both, or neither. Classical base-rate neglect is a failure of the first. A failure of the second is what this paper is about. The cue-density effect from contingency learning is identified as the second neglect: The cue-density effect is the second under-correction. A learner who judges a common cue as stronger than an equally-predictive rare cue is failing to divide by how often the cue occurs.

The paper identifies three literatures describing the same underlying quantity: Three literatures, then, three names, one underlying quantity: divide the overlap by the cue circle, or not. These are base-rate neglect, the cue-density effect, and the contiguity/sensitivity/validity triple from statistical learning.

A central measurement claim is that "Which weight a study can measure depends on the kind of judgment it collects. A graded rating — 'how strongly does this cue predict the outcome' — expresses the whole equation, cue term and all. A two-choice identification — 'given the cue, is it outcome A or outcome B' — compares two such judgments that share the same cue, so the cue term −ρev log P(C) is identical in both and cancels from the comparison."

The paper's sharpest prediction is a double dissociation: "the cue-frequency manipulation should move the cue weight and leave the prior weight alone, and the outcome-frequency manipulation the reverse. A one-parameter account predicts no such split — the two manipulations move a single number together — so the dissociation is a clean test to run. The author states plainly: This experiment has not yet been run: the prediction stands as a prediction."

The paper also shows the cue-frequency weight appears in six standard learning-and-memory models (Bayes/information theory, signal detection, Rescorla–Wagner, MINERVA2, prototype, exemplar/GCM), but "the accounts agree only on the surface... It holds only because the standard design collapses the data onto a 2×2 table, and on that table the cue-frequency weight is the only thing free to vary — so any account rich enough to fit the table is forced to agree there."

The four corners of the weight space recover familiar quantities: validity P(OC) — the calibrated Bayesian at (1,1); lift / causal strength (rating) · the signal-detection contrast log(s/s′) (choice) at (1,0); contiguity — the raw joint, prior intact at (0,1); and sensitivity P(CO) — pure co-occurrence at (0,0).

The paper includes extensive appendices on estimation: A.1 derives the leak from outcome manipulations in closed form (ε ≡ d log P(C) / d log q = 1 − s′/P(C)); A.2 shows omitting the cue term biases the recovered prior weight, with a perfectly calibrated observer (ρ ev = ρ pr = 1) is recovered as ρ̂ pr ≈ 0.33 on the thirteen-cell design; A.3 constructs a design removing the leak; A.4 gives identifiability conditions; A.5 and A.6 provide power analyses showing the planned thirty subjects give 90% power at the planning truth of ρ pr = 0.70.

The paper concludes with a caveat about interpretation: "At the level of mathematics, Bayes' rule contains two logarithmic corrections, one per base rate. That is arithmetic, and it is not in dispute. At the level of a behavioural model, a learner applies each correction partially... At the level of cognitive implementation, this paper says nothing."

Improvements to AI systems:

1. Dual-base-rate calibration module – Add a two-weight correction layer (ρpr, ρev) to any probabilistic classifier or LLM that outputs cue–outcome associations. The system can explicitly track and adjust for both outcome frequency (prior) and cue frequency, preventing overconfidence in predictions from common cues and underconfidence from rare cues.

2. Cue-density bias detector – Implement a diagnostic tool that measures whether an AI system under-divides by cue occurrence. It can flag when a model judges a frequent cue as more predictive than an equally-valid rare cue, and automatically re-weight the cue term.

3. Task-aware weight estimation – Build an AI that infers which base-rate weight is being applied based on the judgment format. For graded ratings, it estimates both ρpr and ρev; for two-choice identifications, it recognizes the cue term cancels and recovers only ρpr. This allows the system to adapt its correction strategy to the output type.

4. Double-dissociation testing framework – Create an experimental harness that manipulates cue frequency and outcome frequency independently. The AI can run the proposed dissociation test, verify whether its own weights move separately (as the two-axis model predicts) or together (as one-parameter models predict), and self-correct if it finds a single-axis failure.

5. Model-agnostic weight-space mapping – Equip AI systems with a module that maps any learned association onto the four corners of the weight space (validity, lift, contiguity, sensitivity). This lets the system report which base-rate correction it is applying, and switch between them for different downstream tasks (e.g., causal reasoning vs. co-occurrence retrieval).

6. Leak-proof estimation routine – Incorporate the appendix’s closed-form leak correction (ε = 1 − s′/P(C)) and the thirteen-cell design fix. The AI can estimate ρpr without contamination from cue-frequency effects, yielding unbiased prior-weight recovery even when cue frequency varies.

7. Power-aware experimental design – Use the paper’s power analyses to let an AI system plan its own learning experiments. It can compute the sample size needed to detect a given ρpr or ρev with 90% power, and decide whether a planned dataset is sufficient to separate the two weights.

What the improved AI system can do:

• Predict outcomes with calibrated confidence that separately accounts for how common the outcome is and how common the cue is, avoiding both classical base-rate neglect and the cue-density effect.

• Diagnose its own biases: it can tell a user whether it is under-weighting the prior, under-dividing by cue frequency, or both, and report the exact ρpr and ρev values.

• Switch between judgment formats (graded rating vs. two-choice) without losing calibration, because it knows which weight is identifiable in each format.

• Run self-experiments to test whether its internal learning rule is one-parameter or two-parameter, and re-train itself if it fails the double dissociation.

• Recover unbiased estimates of prior weight from messy, real-world co-occurrence data, even when cue frequencies vary wildly, using the leak-correction formulas.

• Design its own training sets or active-learning queries to maximize statistical power for separating the two base-rate weights, minimizing the number of samples needed.