COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training
summary
The gist
Optical chemical structure recognition (OCSR) translates molecular images into machine-readable representations like SMILES strings [1] and remains challenging in real-world documents due to…
In short
COMO is a closed-loop framework for Optical Chemical Structure Recognition (OCSR) that uses Minimum Risk Training (MRT) to overcome limitations of standard Maximum Likelihood Estimation (MLE). It mitigates exposure bias by iteratively sampling and evaluating model predictions against molecule-level objectives like chemical validity and structural similarity. This method significantly improves accuracy on real-world chemical diagrams compared to traditional learning approaches.
Key concepts
- Exposure Bias
- This occurs when a model is trained using teacher forcing, meaning it learns to condition on correct ground-truth prefixes during training. However, at inference time, the model must rely on its own previous predictions. This mismatch causes errors made early in decoding to accumulate and propagate incorrectly.
- Minimum Risk Training (MRT)
- MRT is a training strategy that minimizes expected cost by directly optimizing over molecule-level objectives instead of relying solely on token-level MLE. It achieves this by iteratively sampling the model's own predictions, evaluating them using non-differentiable metrics, and updating parameters to favor higher-quality candidates.
- Reward Function Decomposition
- The reward function is split into three parts: chemical validity (is the structure chemically correct?), structural similarity (how close is it to the ground truth?), and exact match (is it identical?). This modular design allows the system to optimize for multiple, complex molecular quality criteria simultaneously.
- Closed-Loop Framework
- COMO creates a loop where the model generates predictions, evaluates them against real molecule metrics (the reward), and uses that feedback to retrain itself. This cycle closes the gap between training data and real-world evaluation, ensuring the model learns from its own performance signals.
Terminology used across episodes
This episode discusses
- COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training · Paper Radio
- Sequence Level Training with Recurrent Neural Networks
- Self-critical Sequence Training for Image Captioning
- OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery
The paper
COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training · Read on arXiv
Zhuoqi Lyu, Qing Ke
City University of Hong Kong
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training".
Jane: Here's the gist of the paper. Optical chemical structure recognition (OCSR) translates molecular images into machine-readable representations like SMILES strings
1: and remains challenging in real-world documents due to inexhaustible variations in chemical structures, shorthand conventions, and visual noise <ref:2604.23546#pg2> Most existing deep-learning-based approaches rely on teacher forcing with token-level Maximum Likelihood Estimation (MLE) which suffers from exposure bias and hinders optimization towards molecular-level evaluation criteria such as chemical validity and structural similarity <ref:2604.23546#pg3> Here we introduce Minimum Risk Training (MRT) to OCSR and propose COMO, a closed-loop framework that mitigates exposure bias by directly optimizing over molecule-level, non-differentiable objectives by iteratively sampling and evaluating the model’s own predictions <ref:2604.23546#pg4> Experiments on ten benchmarks including synthetic and real-world chemical diagrams from patent and scientific literature demonstrate that COMO substantially outperforms existing rule-based and learning-based methods with less training data <ref:2604.23546#pg2> Pruning redundant reasoning tokens COMO is a closed-loop optical molecule recognition framework built with Minimum Risk Training (MRT) which mitigates exposure bias by directly optimizing over molecule-level, non-differentiable objectives by iteratively sampling and evaluating the model’s own predictions <ref:2604.23546#pg2> This framework retains the encoder–decoder backbone while augmenting purely token-level MLE with a joint MLE–MRT training strategy <ref:2604.23546#pg2> The reward function is modular, combining three complementary signals: chemical validity, exact string match, and an interchangeable structural similarity term <ref:2604.23546#pg2> This design offers three practical advantages including closing the exposure-bias loop by computing gradients over samples drawn from the model’s own predictive distribution rather than teacher-forced prefixes <ref:2604.23546#pg2> How the training signal is built The Minimum Risk Training (MRT) loss is defined as LMRT(θ) = E ŷ∼pθ(·x)
2604.235468: which minimizes the expected cost under the model’s own sampling distribution <ref:2604.23546#pg9> The reward function R(ˆy, y∗) is decomposed into three terms capturing finer aspects of prediction quality: R(ˆy, y∗) = wv · 1
valid: + ws · Sim(ˆy, y∗) · 1
valid: + we · 1
exact: <ref:2604.23546#pg10> The similarity component Sim is modular and can be instantiated as edit distance, fingerprint-based Tanimoto similarity, or a learned visual similarity score from a separate image encoder Where the gains come from COMO achieves state-of-the-art exact match accuracy across all ten OCSR benchmarks with particularly large improvements on challenging real-world datasets including JPO (+10.2 pp), UOB (+3.6 pp), ACS (+16.3 pp), and WildMol-10K (+0.3 pp) Ablation studies further confirm that the gains stem from the training mechanism rather than from additional data, as replacing MRT steps with identically scheduled MLE steps on the same real-world images results in performance degradation on 5 out of 10 benchmarks Furthermore, applying MRT to a structurally distinct model, SwinOCSR, yields accuracy gains of 2–20× on real-world benchmarks, confirming that MRT is a model-agnostic training principle for OCSR COMO achieves the highest exact-match accuracy across all 10 benchmarks, with both variants delivering comparable results (within 2 percentage points of each other), suggesting that MRT is robust to the choice of similarity measurement The performance improvement on JPO, USPTO, and ACS is marginal compared to MRT results (lower half of Table II) This pattern indicates that MRT is especially effective at bridging the domain gap between synthetic training images and real-world test images, which cannot be achieved by MLE alone, even when given the same additional data The gains range from modest on synthetic data (+1.1 on Indigo, +2.7 on ChemDraw) to substantial on real-world data (+5.9 on CLEF, +12.2 on JPO, +10.1 on UOB, +14.5 on ACS, +10.7 on WildMol-10K) The work is the first effort to introduce MRT into end-to-end OCSR Page 1 --- COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training Zhuoqi Lyu1 and Qing Ke1, Department of Data Science, College of Computing, City University of Hong Kong, Hong Kong, China (Dated: April 28, 2026 Abstract Optical chemical structure recognition (OCSR) translates molecular images into machinereadable representations like SMILES strings or molecular graphs <ref:2604.23546#pg2> Most existing deep-learning-based approaches rely on teacher forcing with token-level Maximum Likelihood Estimation (MLE) which suffers from exposure bias, as models are trained under ground-truth prefixes but must condition on their own previous predictions during inference <ref:2604.23546#pg3> Moreover, token-level MLE objectives hinder the optimization towards molecular-level evaluation criteria such as chemical validity and structural similarity <ref:2604.23546#pg2> Here we introduce Minimum Risk Training (MRT) to OCSR and propose COMO (Closed-loop Optical Molecule recOgnition), a closed-loop framework that mitigates exposure bias by directly optimizing over molecule-level, non-differentiable objectives, by iteratively sampling and evaluating the model’s own predictions <ref:2604.23546#pg2> Page 2 --- I.
Tom: INTRODUCTION Optical chemical structure recognition (OCSR) aims to convert molecular structure images into machine-readable representations such as linear notations like SMILES strings
1: It is a fundamental component for large-scale chemical information extraction, enabling the digitization of compounds from patents
2, 3: and scientific literature
4: Although recent deep-learning-based OCSR systems have substantially improved over rule-based pipelines by framing the task as image-to-sequence generation or graph prediction, accurate recognition from real-world documents remains challenging <ref:2604.23546#pg2> As presented in Fig.
Jane: 1, models must cope with complex ring structures, substituent abbreviations (e.g., “Et”, “Me”), condensed formulas (e.g., “COOH”), R-group placeholders (e.g., “R1”, “L”), as well as cluttered surrounding text and visual artifacts introduced by scanning and document layout <ref:2604.23546#pg2> These factors create a substantial distribution gap between the clean, synthetic images commonly used for training and the heterogeneous molecular depictions encountered in real documents, causing models to produce predictions that are structurally plausible but chemically incorrect <ref:2604.23546#pg2> A central reason for this failure is the mismatch between how current encoder-decoder OCSR models are trained and how they perform inference <ref:2604.23546#pg2> Most modern systems rely on an autoregressive sequence decoder trained with teacher forcing, which minimizes token-level cross-entropy under ground-truth prefixes <ref:2604.23546#pg2> While effective for learning local syntactic patterns, this Maximum Likelihood Estimation (MLE) objective introduces two compounding limitations: • Exposure bias: Under teacher forcing, the decoder is trained to condition on goldstandard prefixes, whereas at inference time, it must condition on its own previous outputs rather than ground-truth prefixes
5, 6: <ref:2604.23546#pg2> Consequently, errors made early in decoding can propagate and accumulate in ways never encountered during training <ref:2604.23546#pg2> This issue is especially severe for SMILES generation: a single incorrect atom type, misplaced parentheses, or erroneous ring-closure digit can render an otherwise correct molecular structure chemically invalid, turning a near-miss into a complete failure <ref:2604.23546#pg2> • Objective function mismatch: The token-level cross-entropy objective is structurally incapable of optimizing the molecular-level metrics, including chemical validity, Tanimoto similarity, and exact structural match, which the evaluation actually measures <ref:2604.23546#pg2> Page 3 --- CLEF Input Image log p = -34.4 R = 0.41 Greedy Prediction log p = -35.0 R = 1.00 Candidate 1 log p = -54.7 R = 0.42 Candidate 2 log p = -55.0 R = 0.54 Candidate 3 JPO log p = -11.8 R = 0.46 log p = -12.6 R = 0.50 log p = -12.7 R = 0.38 ACS log p = -8.9 R = 0.42 log p = -9.4 R = 0.31 log p = -11.1 R = 0.42 log p = -12.2 R = 1 <ref:2604.23546#pg4> WildMol-10K log p = -25.8 R = 0.44 log p = -47.0 R = 0.41 log p = -52.1 R = 0.44 log p = -66.4 R = 1 <ref:2604.23546#pg4> FIG.
Tom: 1 Examples of predictions made by an MLE-trained model The second column presents greedy-search predictions ˆyi, with the corresponding log probabilities log p(ˆyix) (larger indicates higher probability) annotated below In all shown examples, the greedy predictions fail (highlighted in red), with atom-level discrepancies explicitly highlighted in the skeletal formulas In the third to fifth columns, we display three prediction candidates generated using multinomial sampling (N = 32, temperature = 0.5) We show that the correct predictions (shown in green) can lie within the MLE model’s search space By applying meaningful similarity measurements (Tanimoto similarity is used in this figure) between the candidate and the ground truth, we can compute a reward R (annotated below) that serves as a guiding signal to optimize the decoding distribution backward When fed the same images into our trained COMO model, it always hits the correct predictions with the highest R Page 4 --- FIG.
Title and authors: Tom: So we're talking about the "COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training" paper. The authors are Zhuoqi Lyu and Qing Ke from the Department of Data Science at City University of Hong Kong.
Jane: That's a pretty specific team for such a deep learning problem. What does that title actually tell us about what they did?
Lu: It tells us that they are taking their existing optical molecule recognition system and adding this closed-loop training mechanism, the Minimum Risk Training, to fix the problems we just talked about.
Tom: So it's not just a tweak; it's a whole new way of training the model by making it check its own work against real molecular quality metrics <ref:2604.23546#pg1>.
Jane: And what does this mean for the world? If they can fix the chemical validity issue, what kind of applications do we see?
Lu: It means we can move beyond just making things look plausible to actually ensuring the molecules are chemically correct and structurally sound, which is a huge leap for drug discovery or any chemical information pipeline <ref:2604.23546#pg2>.
Tom: I wonder if this makes it easier to build systems that handle messy, real-world documents, like those from old patents or scientific journals.
Jane: That's right, because they are specifically addressing the visual noise and shorthand conventions that confuse models when they look at real documents <ref:2604.23546#pg2>.
Lu: The paper shows this framework retains the original encoder-decoder backbone but augments the token-level training with a joint MLE and MRT strategy <ref:2604.23546#pg2>.
The paper's summary: Tom: So, to summarize what COMO does, it uses Minimum Risk Training to directly optimize for molecule-level objectives that aren't differentiable, like chemical validity and structural similarity <ref:2604.23546#pg1>.
Jane: That’s the core idea: they’re not just optimizing for how well the model predicts the next token; they are making it optimize for whether the whole final molecule is chemically valid or structurally similar to the target <ref:2604.23546#pg2>.
Lu: The reward function they use is modular, combining three signals: chemical validity, an exact string match, and a structural similarity term that can be swapped out for different kinds of similarity measures <ref:2604.23546#pg1>.
Tom: That modularity is interesting because it means they aren't locked into one definition of "good" structure or "good" representation.
Jane: And the way they build the training signal is key here; they define the Minimum Risk Training loss as minimizing the expected cost under the model’s own sampling distribution <ref:2604.23546#pg9>.
Lu: They decompose that reward into three parts: a weight for validity, a weight for similarity if it's valid, and a weight for exact matches <ref:2604.23546#pg10>.
The paper's improvements: Tom: What are the actual results they show? I want the numbers. How much better is this compared to just using standard MLE training?
Jane: They tested it on ten different benchmarks, including synthetic and real-world chemical diagrams from patents and literature <ref:2604.23546#pg2>.
Lu: The results show that COMO substantially outperforms existing rule-based and learning-based methods, even when they use less training data than previous approaches <ref:2604.23546#pg2>.
Tom: That's good, but how much better are we talking? What kind of improvements on those real-world datasets?
Jane: On ten benchmarks, they saw big jumps. For example, on JPO, they improved by ten point two percent, and on ACS, they saw a jump of sixteen point three percent <ref:2604.23546#pg2>.
Lu: They also did ablation studies that confirm these gains come from the training mechanism itself and not just having more data <ref:2604.23546#pg2>.
Conclusion: Tom: So, to wrap up, the main implication is that this closed-loop training mechanism makes a massive difference when moving from synthetic training images to real-world test images <ref:2604.23546#pg1>.
Jane: It bridges that gap where standard MLE just can't reach because it doesn't understand chemical validity or structural similarity <ref:2604.23546#pg1>.
Lu: The paper shows that MRT is model-agnostic; they tested it on a different model called SwinOCSR and saw accuracy gains of two to twenty percent on real-world benchmarks <ref:2604.23546#pg2>.
Tom: That's compelling, because if it works across different architectures like that, it suggests this is a really robust training principle for optical molecule recognition two thousand six hundred four point zero two five four three.
Jane: So the final picture is that COMO achieves the highest exact-match accuracy across all ten benchmarks, and both variants perform very similarly, within two percentage points of each other <ref:2604.23546#pg1>.
Lu: It's interesting that on some datasets like JPO and ACS, the performance improvement is smaller compared to other datasets like CLEF or UOB <ref:2604.23546#pg1>.
Meng: From an engineering standpoint, it’s good to know that the gains come from the training itself, not just feeding it more data on the same images <ref:2604.23546#pg2>.
Tom: And for deployment at scale, if we can use this MRT approach instead of just relying on token-level MLE, that sounds like a cleaner way to build systems that actually work in the real world two thousand six hundred four point zero two five four three.
More episodes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI
- 2605.23500-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation