Critique of "Use of roster charts in the investigation and prosecution of nurses..." by John O' Quigley

arXiv:2606.29394 · stat.AP · Submitted 2026-06-28 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Critique of "Use of roster charts in the investigation and prosecution of nurses..." by John O' Quigley".

Jane: Here's the gist of the paper. The paper critiques John O’Quigley's analysis of roster charts in investigating nurse misconduct by demonstrating serious statistical errors that undermine his main hypothesis regarding the chart's predictive power.

Tom: The single most informative sentence about the paper's subject and finding is: The roster chart cannot support the hypothesis that a killer nurse is behind the spike in deaths and collapses, let alone that it is Lucy Letby.

Jane: Critique of Statistical Modeling The authors identify several serious problems in O’Quigley’s statistical analyses concerning the roster chart data.

Tom: Specifically, they point out errors in his main modelling assumption regarding the independence of nurses and events.

Title and authors: Tom: To summarize what this critique is really saying, they are pointing out serious statistical errors in O’Quigley’s analysis of the roster chart data concerning nurse misconduct.

Jane: They break down his main modelling assumption, which is that nurses and events are completely independent, and show it's seriously wrong based on the fitted beta-binomial distribution.

Lu: The authors introduce new binary variables Yi and Xi to address the core question of presence given suspicion, focusing on the probability Pr(Yi = oneXi = one) <ref:2606.29394#pg1>.

Meng: They argue that this specific conditional probability is what actually needs to be calculated to answer the relevant questions about presence.

Tom: But they don't just stop there; they look at the visual presentation, suggesting Figure one itself is misleading and should be viewed skeptically by statistical experts <ref:2606.29394#pg1>.

Jane: They highlight how intuition can cause errors, noting that a nurse working more shifts might appear more likely to be present during a suspicious event than another nurse on the chart.

Lu: The paper shows that the model used by O’Quigley exhibits underdispersion, not overdispersion, when analyzing the thirty-seven random nurses' random column totals in a twenty-five by thirty-seven roster chart <ref:2606.29394#pg3>.

Meng: That underdispersion result is key because it means his statistical model simply doesn't fit the actual data structure of the roster chart at all.

Tom: So, to put it plainly, the empirical estimate for the suspected nurse’s presence being one hundred percent has no value because of this selection bias in how he calculated it.

Jane: It implies that predictions made using O’Quigley’s model fit are essentially useless because the model fails to match what is actually happening in those twenty-five shifts <ref:2606.29394#pg3>.

Lu: Ultimately, the authors conclude by agreeing with O’Quigley that from a forensic statistical point of view, the roster chart is fake evidence and shouldn't be shown to jurors.

Meng: They suggest that confirmation bias and poor communication between O’Quigley and his programmer seem to have led to this whole catastrophe.

Tom: So, the main takeaway is that you can't trust a roster chart just because someone puts a complex statistical model on it; the data itself has serious structural issues.

The paper's summary: Jane: Now, moving into what they suggest for improvement, the authors propose several ways to make this analysis more rigorous and statistically sound than O’Quigley did originally.

Tom: First, they recommend improving the statistical modeling by replacing O’Quigley's assumption of independence with a beta-binomial model that accounts for heterogeneity among nurses.

Lu: They argue this beta-binomial model shows that the actual sample variance of the thirty-seven innocent nurse's twenty-five shift presence totals in the actual roster chart is smaller than the sample mean <ref:2606.29394#pg3>.

Meng: So, instead of assuming everything is random and independent, you use a distribution that acknowledges nurses are actually different from one another.

Tom: That’s a big mechanical change because it directly addresses why his initial model didn't fit the observed data variance correctly.

Jane: Second, they suggest enhancing the forensic statistical analysis by implementing a more rigorous approach to calculating Pr(Yi = oneXi = one) <ref:2606.29394#pg1>.

Lu: This conditional probability is defined as the probability of presence given that an event is deemed suspicious, which would allow for a comparison between the suspect and any randomly chosen nurse.

Meng: That means instead of just looking at one nurse's presence, you calculate how much more likely that specific nurse is to be there compared to a random person on the roster.

Tom: And third, they propose developing a system capable of identifying selection bias in evidence interpretation using Monte Carlo simulations based on the estimated beta distribution.

Lu: This simulation would allow an AI to test hypotheses about which nurse could have been selected as "the common denominator" when trying to simulate the data-generating mechanism behind Figure one assuming innocence for all nurses <ref:2606.29394#pg1>.

Jane: It sounds like they want a way to run thousands of simulations under the assumption of no guilt to see what kind of patterns naturally emerge in the data.

Tom: It’s about using simulation not just as a check, but as a way to actively test the assumptions about how these charts are constructed before you draw any conclusions.

The paper's improvements: Tom: So, wrapping up this critique of "Use of roster charts in the investigation and prosecution of nurses," the main implication is that we need much stricter statistical validation for forensic statistics used in legal cases.

Jane: It means that just because a chart is complex or visually appealing doesn't make it reliable evidence if the underlying statistical assumptions are fundamentally broken.

Lu: The paper strongly supports O’Quigley's view that the roster chart, from a forensic perspective, is fake evidence and should not be shown to jurors.

Meng: From an operational standpoint, it suggests that relying on these kinds of charts for criminal investigation carries a very high risk of misinterpreting the data due to inherent biases.

Tom: The paper really shows how confirmation bias can lead people down a path where they accept a model that just doesn't fit the actual reality of the scheduling data.

Jane: It’s a call for caution in using visual evidence when dealing with complex, real-world data like hospital rosters, which is something we see everywhere.

Lu: The authors are essentially arguing that without rigorous methods like the proposed beta-binomial modeling and Monte Carlo simulations, these kinds of analyses are scientifically unsound.

Meng: They emphasize that the method itself needs to be tested against real traffic or at least a much more robust simulation environment before it can be deployed in any serious context.

Tom: So, as we look at this paper, we see a clear path toward requiring better statistical mechanics when we try to use complex scheduling data to make life-or-death decisions in investigations.

Jane: And that means listeners should keep asking questions about the assumptions behind the charts they see in the news or on TV.

Lu: The paper itself is a strong argument for demanding higher standards of statistical proof before evidence is presented in court.

Conclusion: Tom: So we just finished looking at this critique of John O’Quigley's paper, "Critique of 'Use of roster charts in the investigation and prosecution of nurses'". The main takeaway is that the roster chart itself is statistically suspect evidence and can't support a hypothesis about nurse misconduct.

Jane: That’s right. They show serious errors in O’Quigley’s modeling assumptions, specifically how he treated the independence of nurses and events on that chart.

Lu: It really hammers home that his fundamental assumption—that nurses are completely independent—is seriously wrong because the beta-binomial distribution analysis shows underdispersion instead of overdispersion.

Meng: From a deployment standpoint, if you rely on a model that doesn't fit the actual data structure at all, you can't deploy it in production; it just breaks.

Lalam: The authors are suggesting that the empirical estimate of presence being one hundred percent for the suspected nurse has no value because of selection bias in how he calculated it.

Tom: I agree. It’s not just that the numbers look wrong, but the entire foundation of his prediction is flawed because the model doesn't match what's actually happening on that roster chart.

Jane: And they point out how intuition leads to errors too; for example, a nurse working more shifts might seem more likely to be present during a suspicious event than another nurse on the list.

Lu: They also show how O’Quigley sampled the missing rows using the other nurses' relative frequencies, which is where the biggest statistical leap and potential error happens.

Meng: That part about sampling with maximum likelihood estimation—it sounds like a huge assumption that needs a lot more testing on real traffic before it's considered valid.

Lalam: The paper suggests that predictions made from O’Quigley’s model fit are useless because the fitted model simply fails to fit the actual data at all.

Tom: It really confirms what they said: the roster chart is a scientific fake and confirmation bias led to a catastrophe in that investigation.

Jane: It makes you think about how much we need to check these kinds of visual analyses before someone tries to use them as proof in court.

Lu: The improvements suggested, like using a beta-binomial model that accounts for nurse heterogeneity, are the way forward for making these kinds of forensic statistical investigations actually work.

Meng: If we were building a system now, we’d absolutely need those Monte Carlo simulations to see how selection bias affects the results before we even think about deploying it.

Lalam: The future work they mention is using these methods to rigorously test the conditional probability Pr(Yi = oneXi = one) between the suspect and a randomly chosen nurse.

Tom: It’s a good way to establish whether that probability actually differs in any significant way between the person under suspicion and just anyone else on that roster.

Jane: So, we've seen how this paper critiques O’Quigley's work on "Use of roster charts in the investigation and prosecution of nurses". It leaves us with a clear warning about trusting statistical models if they ignore the underlying data structure.

Lu: We need to be careful when we look at these types of charts because the method itself can introduce biases that make the results completely unreliable.

Meng: The cost of running a model that's fundamentally misaligned with reality is just wasted compute cycles and wasted time for everyone involved.

Lalam: We’ll keep an eye on how researchers tackle these kinds of evidence interpretation issues because they are really important for understanding how we use data in real-world scenarios.

Leiden University · National Crime Agency

stat.AP

Submitted: 2026-06-28

Updated: 2026-10-29

Comments: First revision. Appendix with some R code added

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 57/100

The gist: The paper critiques John O’Quigley's analysis of roster charts in investigating nurse misconduct by demonstrating serious statistical errors that undermine his main hypothesis regarding the chart's

Key concepts

Independence Assumption: O'Quigley assumed nurses and shifts were completely independent. The authors prove this assumption is wrong by showing the data exhibits underdispersion, meaning the actual variance of presence totals is smaller than expected, contradicting his model.
Beta-Binomial Distribution: This statistical model was used to analyze the roster chart. The authors show that applying this model reveals a pattern (underdispersion) that invalidates O'Quigley's initial statistical framework for analyzing nurse presence.
Selection Bias: The estimated probability of presence for the suspected nurse is considered meaningless due to selection bias. This occurs because the analysis ignores how the data was chosen, leading to an inflated estimate of 100% presence that is not representative of reality.

Terminology

Selection Bias — 6× in this paper · explained in 6 episodes
Beta-Binomial Distribution — 5× in this paper · explained in 2 episodes
Composition — 3× in this paper · explained in 2 episodes
Editor — 2× in this paper · explained in 2 episodes
Core — 1× in this paper · explained in 2 episodes

Summary

The paper critiques John O’Quigley's analysis of roster charts in investigating nurse misconduct by demonstrating serious statistical errors that undermine his main hypothesis regarding the chart's predictive power.

The single most informative sentence about the paper's subject and finding is: The roster chart cannot support the hypothesis that a killer nurse is behind the spike in deaths and collapses, let alone that it is Lucy Letby.

Critique of Statistical Modeling

The authors identify several serious problems in O’Quigley’s statistical analyses concerning the roster chart data. Specifically, they point out errors in his main modelling assumption regarding the independence of nurses and events.

The authors introduce binary variables, Yi and Xi, to address the central question of presence given suspicion. They state that the quantity Pr(Yi = 1Xi = 1) is what will enable them to address the relevant questions.

Flaws in Data Interpretation

The paper highlights several misleading aspects of the visual presentation of the data, noting that Figure 1 is described as a fake by a meeting of the Legal Section of the Royal Statistical Society. The authors show that intuition can lead to errors, as nurse working the most shifts might be more likely to be present when a suspicious event occurs than nurse 23.

The paper argues that the visual impression ignores bias and can lead to incorrect conclusions, noting that ignoring these biases could easily lead to concluding that the probability of being present is of the order 100%.

Model Assumptions and Overdispersion

The critique focuses heavily on O’Quigley’s fundamental assumption that nurses and shifts are completely independent. The authors demonstrate that the model used by O’Quigley has been proven to be seriously wrong.

The analysis of the fitted beta-binomial distribution shows that the resulting probability distribution of 37 random nurses’ random column totals in a 25 × 37 roster chart would exhibit underdispersion, not overdispersion.

Conclusion on Validity

The authors conclude that the empirical estimate of presence for the suspected nurse—at 100%—has no value due to selection bias. They assert that predictions made from O’Quigley’s model fit are useless because the fitted model completely fails to fit to the actual data.

The paper ultimately agrees with O’Quigley that the roster chart is a scientific fake. The confirmation bias and poor communication between O’Quigley and his programmer seem to have led to a catastrophe.

References

Knapton S and Elston P (2024) Spike in deaths at letby hospital ’could be explained by how small and premature babies’; and: The scientific case against lucy letby (part two): the explicable spike.

O’Quigley J (2024) Logical and statistical errors in the investigation and prosecution of suspected serial killer nurses.

O’Quigley J (2025) Use of roster charts in the investigation and prosecution of nurses suspected of inflicting deliberate harm on patients. Medicine, Science and the Law.

Rubin DB (1981) The Bayesian Bootstrap. The Annals of Statistics 9(1): 130 – 134.

--- Page 5 ---

The paper is titled Critique of “Use of roster charts in the investigation and prosecution of nurses by John O’ Quigley" and it explores an interesting hypothesis concerning statistical information hidden in the part of the infamous Lucy Letby roster chart pertaining to the 37 other nurses.

The authors point out some serious errors in his statistical analyses because the data actually contains information which strongly disproves his main modelling assumption. They strongly agree with him that from a forensic statistical point of view, the roster chart is fake evidence which should not have been shown to jurors.

The introduction sets up the idea that such a chart could have been constructed for any other nurse who worked a large number of shifts, if the characterisation of events happening during shifts as suspicious has been biased by suspicion of that particular nurse.

The authors detail the process where hospital investigators nominated a total of 61 shifts as containing potentially suspicious events. They explain that Lucy Letby is seen as the common factor in these 25 events.

The authors state that O’Quigley’s fundamental assumption (assuming Lucy’s innocence) is that the X’s in the original 61 row spreadsheet, after randomly shuffling the order of the rows, are all placed at random, independently of one another, but using a different probability of being rostered for each of the 38 nurses.

The authors hypothesize that O’Quigley has sampled the 36 missing rows of the table by using the 37 other nurses’s observed relative frequency of presence in the columns of the 25 observed rows, and done maximum likelihood estimation of the beta-binomial using the augmented table of 61 rows.

The authors hypothesize that O’Quigley has sampled the 38 missing rows of the table by using the 37 other nurses’s observed relative frequency of presence in the columns of the 25 observed rows, and done maximum likelihood estimation of the beta-binomial using the augmented table of 61 rows.

The authors state that O’Quigley has sampled the 38 missing rows of the table by using the 37 other nurses’s observed relative frequency of presence in the columns of the 25 observed rows, and done maximum likelihood estimation of the beta-binomial using the augmented table of 61 rows.

Improvements for AI systems

improve statistical modeling of nurse presence by replacing O’Quigley's assumptions about independence with a beta-binomial model that accounts for heterogeneity, as this model shows that the actual sample variance of the 37 innocent nurse’s 25 shift presence totals in the actual roster chart is smaller than the sample mean.
enhance forensic statistical analysis by implementing a rigorous approach to calculating Pr(Yi = 1Xi = 1), which is defined as the probability of presence given that an event is deemed suspicious, to determine if this probability differ[s] in any significant way between the suspect and a randomly chosen nurse.
develop a system capable of identifying selection bias in evidence interpretation by using Monte Carlo simulations based on the estimated beta distribution, allowing the AI to simulate the data-generating mechanism behind Figure 1 as follows (assuming innocence of all nurses) to test hypotheses about which nurse could have been selected as the common denominator.

Abstract

The paper "Use of roster charts in the investigation and prosecution of nurses suspected of inflicting deliberate harm on patients" by Prof. John O'Quigley explores an interesting hypothesis concerning statistical information hidden in the part of the infamous Lucy Letby roster chart pertaining to the 37 other nurses. Unfortunately, we have to point out some serious errors in his statistical analyses. The data actually contains information which strongly disproves his main modelling assumption. We do, however, strongly agree with him that from a forensic statistical point of view, the roster chart is fake evidence which should not have been shown to jurors.