Methodological Pitfalls in Predicting Ship Turnaround Time at Brazilian Ports: An Empirical Audit and a Reproducible Pipeline
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Methodological Pitfalls in Predicting Ship Turnaround Time at Brazilian Ports".
Jane: Here's the gist of the paper. Two recent studies on ship turnaround time at Brazilian ports report models with R-squared scores above 0.80 on data from the national waterway transportation agency.
Tom: We audit their published datasets and reveal these results are inflated by structural data leakage.
Jane: Specifically, 97 percent of berthing events in their random row-level split appear in both training and testing partitions.
Tom: We demonstrate that a model’s R-squared drops from 0.82 to 0.20 when the data is properly split at the berthing level.
Title and authors: Tom: So we're looking at this paper called "Methodological Pitfalls in Predicting Ship Turnaround Time at Brazilian Ports: An Empirical Audit and a Reproducible Pipeline." It sounds pretty technical, so Jane, can you tell us what the main point is without getting bogged down in the details?
Jane: Well, basically, they are taking two existing studies that claimed they had really good models with R squared scores over zero point eight on data from the national waterway transportation agency. But this paper audits those datasets and shows that those high scores are actually inflated because of something called structural data leakage.
Tom: Data leakage? That means the training data accidentally knows too much about the test data, right? Like, if you randomly split your rows, you might end up with the same event in both groups.
Jane: Exactly. The paper found that ninety-seven percent of berthing events that are in their random row-level split actually show up in both the training and testing partitions. That’s a massive amount of leakage.
Lu: This means their reported performance numbers, like those R squared scores above zero point eight, aren't real performance metrics on new data; they're just artifacts of how they split the data <ref:2611.14713#pg1>.
Meng: From a deployment standpoint, if your model is trained on leaked information, it will look great in testing but fail completely when you put it into production with real, unseen data. It’s not scalable or reliable at all <ref:2611.14713#pg2>.
Tom: So, they are showing us how to fix that initial problem by properly splitting the data at the berthing level instead of just randomly splitting every single row.
Jane: Right, and beyond that basic fix, they found other things going wrong with the original models too, like survivorship bias and undocumented target truncation.
Lu: They also pointed out variables in the original models that mathematically reconstruct the target variable <ref:2611.14713#pg3>.
Meng: That’s bad because you end up building a model that is just memorizing the data structure instead of actually learning a useful pattern for predicting time <ref:2611.14713#pg3>.
The paper's summary: Tom: So, what's the core proposal they are making to fix all these issues? What’s the big picture fix described in this paper?
Jane: The main proposal is a reproducible pipeline. They suggest you aggregate everything to the berthing level first. This means one row represents one ship visit to a berth, which fixes that ninety-seven percent leakage we talked about earlier <ref:2611.14713#pg3>.
Lu: They take the raw data and sum up numeric cargo attributes and use weighted one-hot sums or first operations for things that are categorical <ref:2611.14713#pg3>. That’s how they handle the data grain issue.
Meng: And for those high cardinality text variables, like the cargo group, they don't just drop them; they use multilingual sentence embeddings weighted by cargo mass <ref:2611.14713#pg2>.
Tom: Wait, so they are using these embeddings to capture the meaning of the cargo instead of just treating it as a random label? That seems like a smart way to handle that complexity.
Jane: It is. They use conditional imputation too, which means instead of just deleting rows with missing values, they apply rules based on what those missing values mean structurally <ref:2611.14713#pg3>.
Lu: And the results from this new pipeline are pretty interesting; they tested six different time targets and thirty random seeds, and cargo embeddings showed statistically significant improvements on four of those operational targets <ref:2611.14713#pg2>.
Meng: Four out of six operational targets improved significantly, but the administrative waiting times didn't see any change at all. That’s a key finding about where the signal is strongest <ref:2611.14713#pg2>.
Tom: So what does that mean for us when we try to predict turnaround time? It suggests that how you represent the cargo really matters for what part of the process you are predicting.
The paper's improvements: Jane: The paper details how this new pipeline actually improves things, and it’s not just one thing; it’s a whole set of changes <ref:2611.14713#pg2>. They showed that using the semantic cargo representations yields statistically significant gains on four operational targets.
Lu: Specifically, adding the cargo group embedding yielded a statistically robust improvement on TStay, which is the time at stay, with a difference of one point three three hours and a p value less than ten to the power of negative seven <ref:2611.14713#pg2>.
Tom: one point three three hours. That’s tangible data. But are these improvements consistent across all time predictions, or is it just better at one specific thing?
Jane: It isn't uniform; the results show that cargo embeddings are specific to each target metric rather than being the same across the whole stay process <ref:2611.14713#pg2>.
Meng: So, cargo information helps predict things like TStay, wait to moor time, time moored, and operation time <ref:2611.14713#pg2>, but it doesn't help much with the administrative ones like wait start operation or wait unmooring <ref:2611.14713#pg2>.
Lu: That pattern is interesting because it suggests that cargo identity has a very large and statistically robust effect on the operational components of the stay, like loading and waiting to moor <ref:2611.14713#pg2>.
Tom: So, in simpler terms, if you want to cut down physical time at port—like loading or waiting to moor—focus your cargo data efforts there because that's where the model actually sees the biggest benefit.
Conclusion: Jane: So to wrap up, the main message of this paper is that relying on simple random splits and basic feature handling leads to misleading performance numbers when predicting ship turnaround time at Brazilian ports. The authors show you how to build a proper pipeline using aggregation, conditional imputation, and semantic cargo embeddings <ref:2611.14713#pg2>.
Tom: They also highlighted that the combination of the data release and their original methodology choices produced numbers whose interpretation as predictive performance is misleading, so you need this corrected pipeline to see what's actually happening <ref:2611.14713#pg2>.
Lu: The paper supports a holistic management approach where cargo data is useful for physical planning, but managing administrative performance requires a new approach to process management <ref:2611.14713#pg2>.
Meng: I’d add that the authors suggest managers should look at operational indicators that the original model missed, like document processing status and inspection requirements <ref:2611.14713#pg2>.
Lalam: I'm confident in this direction; semantic cargo representations facilitate more informed predictions regarding waiting time, time at berth, handling time, and total turnaround time <ref:2611.14713#pg2>.
Tom: So we’ve seen how structural leakage ruins the numbers and what a fixed pipeline looks like when you properly aggregate cargo to the berthing level <ref:2611.14713#pg3>. That's all for this paper.
Jane: We’ll be back after the break with another look at some of these other interesting papers on arXiv.
Eduardo Caruso Barbosa Pacheco, Reynaldo Pereira Martins, Alexandre dos Santos Gualberto, Joao Vitor Souza ğ Germano, Milton Miranda Neto, Francisco Louzada
Institute of Mathematical and Computer Sciences, University of Sao Paulo
stat.AP
Submitted: 2026-11-10
Updated: 2026-11-10
Comments: 13 pages, 8 tables and 5 figures
Code: https://github.com/eduardocbpacheco/ANTAQ
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 92/100
The gist: Two recent studies on ship turnaround time at Brazilian ports report models with R-squared scores above 0.80 on data from the national waterway transportation agency.
Key concepts
- Structural Data Leakage
- This occurs when information from the testing data accidentally appears in the training data. In this paper, it was found that 97% of berthing events in the test set were already present in the training set under a random split, causing models to appear highly accurate when they are not truly predictive.
- Berthing-Level Aggregation
- Instead of analyzing data at the individual row level, this method groups all cargo attributes for a single berthing event together. This ensures that each data point in the model represents one complete ship stay rather than just one piece of cargo information, which fixes the leakage issue.
- Cargo Embeddings
- These are numerical representations (vectors) created from complex text descriptions of cargo, weighted by its mass. They allow a model to understand the semantic meaning of cargo—like destination or handling requirements—and use this rich context to make more accurate predictions for specific time targets.
- Target-Specific Heterogeneity
- This means that the impact of a variable (like cargo) is different depending on which prediction you are trying to make. The paper shows that cargo embeddings significantly improve predictions for operational times (like loading), but have little effect on administrative times, highlighting this crucial difference.
Terminology
Summary
Two recent studies on ship turnaround time at Brazilian ports report models with R-squared scores above 0. p. 080 on data from the national waterway transportation agency. p. 0 We audit their published datasets and reveal these results are inflated by structural data leakage. p. 0 Specifically, 97 percent of berthing events in their random row-level split appear in both training and testing partitions. p. 0 We demonstrate that a model’s R-squared drops from 0. p. 082 to 0.20 when the data is properly split at the berthing level. p. 0 We further document additional methodological flaws, including survivorship bias, undocumented target truncation, and the retention of variables that algebraically reconstruct the target. p. 0 To address these issues, we propose a reproducible pipeline that aggregates cargo to the berthing level, utilizes conditional imputation, semantic embeddings weighted by cargo mass, and represents high-cardinality cargo variables through weighted averages of multilingual sentence embeddings. p. 0 Evaluating six time targets and thirty seeds, our pipeline reveals that cargo embeddings yield statistically significant improvements on four operational targets, while administrative waiting times remain unaffected. p. 0 Prior works missed this per-target heterogeneity by conflating genuine signal with leaked identity. p. 0
Pruning redundant reasoning tokens
The paper introduces an empirical audit of published models on Brazilian port data to expose structural data leakage and propose a corrected, reproducible pipeline. This audit quantifies leakage at the row level and identifies four additional issues not previously catalogued. p. 0 The authors demonstrate that structural leakage is present because 97% of test-set berthings appear in the training set under a random split. p. 0
How the training signal is built
The proposed reproducible pipeline involves several rigorous steps to ensure data integrity, starting from raw ANTAQ tables. The process begins with aggregation at the berthing level, where numeric cargo attributes are summed and categorical attributes are handled via weighted one-hot-sum or first operations. p. 5 High-cardinality text variables like cargo group are represented through multilingual sentence embeddings aggregated by cargo mass. p. 7 Conditional imputation is used instead of row removal, applying rules based on the structural meaning of missingness for different variable types. p. 5
Where the gains come from
The significant performance improvements are attributed to using semantic cargo representations, which yield statistically significant gains on four operational targets. Specifically, adding the Cargo Group embedding (M2) yielded a statistically robust improvement on TStay (∆RMSE = +1. p. 833 h, d = 11.83, p < 10−7) and TWaitMooring (+1. p. 225 h, d = 15.41), while the containerized cargo embedding alone did not improve performance significantly on any target. p. 9
The analysis reveals that cargo identity has very large and statistically robust effect on the operational components of the stay (loading and the wait to moor) and no measurable effect on the administrative ones. This pattern is operationally ideal because characteristics such as volume, destination, and handling requirements are linked to demand for berths, loading and unloading operations. p. 9
The results show that cargo embeddings are specific to each target metric rather than being uniform across the vessel’s stay process. Cargo information helps improve predictions for TStay, TWaitMooring, TMoored, and TOperation, though it is less effective for TWaitStartOp or TWaitUnmooring. p. 9
The discrepancy between operational and administrative targets arises because cargo characteristics have a stronger impact on operations than on administrative activities following berthing. The authors conclude that cargo information is useful for physical planning, but managing administrative performance requires a new approach to process management. p. 9
The authors state that the combination of data-release and methodology choices produces numbers whose interpretation as predictive performance is misleading. The corrected pipeline produces performance estimates that are more representative of the underlying problem. p. 0
The final conclusion emphasizes that the combination of data-release and methodology choices produces numbers whose interpretation as predictive performance is misleading. The corrected pipeline makes visible a target-specific structure that a single-target analysis does not capture. p. 2 p. 2 p. 2 p. 2
The paper supports a holistic management approach where cargo data is useful for physical planning, but managing administrative performance requires a new approach to process management. The authors suggest that managers should utilize operational indicators currently missing from the model such as document processing status and inspection requirements.
The paper notes that the outlier filter is a single explicit conjunction, suggesting a more refined version could partition extreme stay-time regimes into their own model. The authors caution that generalizing the cargo-based model into a universal solution for addressing delays is not ideal.
The paper concludes that semantic representations of the cargo can facilitate more informed predictions regarding waiting time, time at berth, handling time, and total turnaround time. These predictions support berth allocation management while also aiding in supply and supply chain management. p. 9
The authors assert that the combination of data-release and methodology choices produces numbers whose interpretation as predictive performance is misleading. The corrected pipeline makes visible a target-specific structure that a single-target analysis does not capture.
The authors suggest that managers seeking to reduce timeframes should utilize operational indicators currently missing from the model such as document processing status and inspection requirements. The analysis supports a holistic management approach where cargo data is useful for physical planning, but managing administrative performance requires a new approach to process management. p. 9
The authors conclude that the combination of data-release and methodology choices produces numbers whose interpretation as predictive performance is misleading. The corrected pipeline makes visible a target-specific structure that a single-target analysis does not capture.
The authors suggest that cargo embeddings are specific to each target metric rather than being uniform across the vessel’s stay process. Cargo information helps improve predictions for TStay, TWaitMooring, TMoored, and TOperation. p. 9 The authors suggest that managers seeking to reduce timeframes should utilize operational indicators currently missing from the model such as document processing status and inspection requirements.
The paper concludes that the combination of data-release and methodology choices produces numbers whose interpretation as predictive performance is misleading. The corrected pipeline makes visible a target-specific structure that a single-target analysis does not capture.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
The system can be improved by implementing a reproducible pipeline that aggregates cargo to the berthing level, as described in Section VI A. This aggregation ensures that each row represents a single berthing event, addressing the structural leakage where
97 percent of berthing events in their random row-level split appear in both training and testing partitions.
-
The AI system can incorporate conditional imputation instead of simply removing rows for missing values, following the rule: "numeric variables for which absence means non-applicability (TEU, ContainerWeight, region/river/waterway financial values) are imputed with zero; numeric variables for which absence means missingness (port coordinates, gross weight) are imputed with the median conditional on the port."
-
The system can utilize multilingual sentence embeddings to represent high-cardinality cargo variables by using a weighted average:
Ea = P i wi e(ci), where e(·) is the embedding and wi is the gross mass of cargo i,
which allows for a representation whereA berthing dominated by one mass-heavy commodity is represented near that commodity in embedding space; a mixed berthing lies between its components weighted by mass.
-
The system can incorporate a semantic outlier filter to handle extreme stay times, applying the rule: "100% of berthings with TStay > 2,000 h satisfy the conjunction NavigationType = 'Port Support' AND OperationType = 'Bunkering' AND TStay > 1,000 h.
This removes the
bimodal tail that otherwise dominates the variance of the regression targets." -
The system can decompose ship turnaround time prediction into six separate regression problems: TStay, WaitMooring, Moored, WaitStartOp, Operation, and WaitUnmooring. This approach reveals
heterogeneous target-specific effects of cargo representation,
showing thatcargo identity matters substantially for the operational components of the stay (loading and the wait to moor) and is not measurable for the administrative ones.
-
The system can perform a controlled ablation over 30 seeds, 6 targets, and 4 configurations to assess which cargo embeddings yield statistically significant improvements. This allows for an evaluation of
target-specific effects of cargo representation,
confirming thatcargo embeddings are specific to each target metric, rather than being uniform across the vessel’s stay process.
Sources
- Entity Embeddings of Categorical Variables
- How to avoid machine learning pitfalls: a guide for academic researchers
Related papers
- Unraveling time-varying causal effects of multiple exposures: integrating Functional Data Analysis with Multivariable Mendelian Randomization
- Strategic Causal Policy Learning with Manipulable Eligibility
- BayesJudge: Uncertainty-Aware Bayesian Meta-Evaluation of Human and LLM Judgments
- Automating structural reliability analysis with a multi-agent large language model framework
- Improving Minority Population Sampling with BISG Probabilities: Evidence from a Survey of Jewish Americans
- When Do Sports Leagues Stop Being Competitive? Bayesian Detection of Competitive-Balance Eras from Match-Level Data