Class-Conditional Conformal Prediction for Signal Temporal Logic Inference with Many Classes

arXiv:2610.14352 · cs.LG · Submitted 2026-10-09 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Class-Conditional Conformal Prediction for Signal Temporal Logic Inference with Many Classes".

Jane: The gist: This work introduces a two-stage procedure combining discriminative formula learning, tightness refinement,

Tom: First, who's behind it and why it matters.

Title and authors: Jane: So if you look at their summary of "Class-Conditional Conformal Prediction for Signal Temporal Logic Inference with Many Classes," it sounds like they’re building a framework that gives us a coverage guarantee when we use these learned formulas to predict the class.

Tom: That's the core idea. They show that under intra-class exchangeability, this method provides a class-conditional guarantee, specifically stating that the probability of being in the predicted set given you are actually in a certain class is greater than or equal to one minus alpha, where alpha is your chosen confidence level.

Lu: The key detail here is how they handle the calibration data. They point out that when you have limited calibration samples per class, those specific thresholds get less precise, and the coverage becomes more variable across different calibration sets because it's a random variable over that data.

Meng: That sounds like a real deployment headache. If the coverage varies too much depending on which small subset of data you use for calibration, how reliable is that guarantee actually in production?

Lalam: The paper addresses this by using the decoding loss as a nonconformity score and calibrating it separately for each class. This is what allows them to achieve a finite-sample class-conditional coverage under intra-class exchangeability.

The paper's summary: Tom: So, to sum up what this paper does, they introduce a two-stage procedure to tackle the gap between classification accuracy and logical specification adherence in multiclass STL inference. They quantify this gap using the positive satisfaction rate and codeword satisfaction rate.

Jane: It’s about moving past just looking at how accurate the classification is, because that doesn't guarantee the inferred formulas are correct according to the true class structure. It makes us measure two things: how well those formulas satisfy what they need to satisfy, and whether they align with what a complete class codeword requires.

Lu: The methodology involves learning discriminative formulas first using a margin-based objective, and then performing tightness refinement to adjust the predicate thresholds toward a small positive robustness target while keeping the formula structure and temporal intervals exactly as they are.

Meng: That sounds like a lot of coupled optimization objectives there. What does that "tightness refinement" actually achieve in practice compared to just using standard margin-based learning?

Lalam: The tightness refinement objective is designed to pull the satisfied required formulas toward a small positive robustness target, while satisfaction, ranking, and anchor penalties limit the loss of satisfaction, class separation, and threshold drift.

The paper's improvements: Tom: So what are the actual improvements they propose? The main thing is that they show how this refinement procedure exposes a controllable tradeoff between moving satisfied outputs toward the STL boundary and limiting losses in satisfaction and class separation.

Jane: And because they use the decoding loss as a nonconformity score calibrated separately for each class, it allows them to provide that finite-sample class-conditional coverage guarantee under intra-class exchangeability. It fixes that issue where pooled calibration might hide undercoverage problems.

Lu: They also show that the magnitude of attainable tightening depends on the data, meaning how much you can actually tighten the formulas isn't fixed and depends on what you're training on.

Meng: So, in terms of deployment, this means we have to be mindful of where we are deploying because the level of tightness you get is tied to your specific dataset. That’s a practical constraint.

Lalam: The paper explicitly shows that even when robustness signs fail to realize the intended class codeword, the positive satisfaction rate and codeword satisfaction rate make this discrepancy very explicit for anyone looking at these results.

Conclusion: Tom: So to wrap up this paper on "Class-Conditional Conformal Prediction for Signal Temporal Logic Inference with Many Classes," the main implication is that we can have accurate multiclass decoding even if the robustness signs don't perfectly match the intended class codeword, because of how they quantify satisfaction.

Jane: The refinement procedure is what exposes a controllable tradeoff between getting those satisfied outputs close to the STL boundary and managing the penalties for losing satisfaction or class separation during training. It’s a way to make sure our formulas are robust in a structured way.

Lu: The research shows that while we can get these guarantees, the extent of how much we can actually tighten the specifications is dependent on the data itself, which is an important limitation to keep in mind for future work.

Meng: For production, this means we have to be careful about tuning those penalties—the satisfaction and ranking penalties—because they directly control how tight your resulting model's temporal constraints are.

Lalam: Ultimately, the paper provides a method that uses class-specific thresholds calibrated via a nonconformity score derived from the decoding loss, which gives us finite-sample class-conditional coverage under intra-class exchangeability for STL inference.

Tom: That’s it for this one. We’ll keep an eye on how this refinement process translates to real deployment scenarios next time we talk about model robustness.

Yixuan Wang, Danyang Li, Matthew Cleaveland, Roberto Tron, Mingyu Cai

University of California, Riverside · Boston University · MIT Lincoln Laboratory

cs.LG

Submitted: 2026-10-09

Updated: 2026-10-09

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: The gist: This work introduces a two-stage procedure combining discriminative formula learning, tightness refinement, and class-conditional conformal prediction to provide finite-sample

Key concepts

Signal Temporal Logic (STL) Inference: This technique uses interpretable logical formulas to classify time series data. Instead of just outputting a label, it outputs a set of rules that describe the temporal patterns of the input sequence. These rules are then used to make a classification decision.
Positive Satisfaction Rate (PSR): This metric quantifies how well the inferred STL formulas satisfy the requirements for each class. It measures if the formulas correctly identify positive events associated with that specific class, ensuring they align with what is expected for that class.
Codeword Satisfaction Rate (CSR): This measures consistency between the learned formulas and a complete codeword representing all possible classes. It checks if the inferred rules are consistent with the entire set of possible outcomes, ensuring comprehensive coverage.
Class-Conditional Conformal Prediction: This is a statistical method used to provide confidence intervals or prediction sets that are tailored specifically for each class. By calibrating the prediction score separately for every class, it provides a robust guarantee on how often the true class will be included in the predicted set.

Terminology

Conformal Prediction — 1× in this paper · explained in 8 episodes

Summary

The gist: This work introduces a two-stage procedure combining discriminative formula learning, tightness refinement, and class-conditional conformal prediction to provide finite-sample class-conditional coverage guarantees for Signal Temporal Logic inference in multiclass settings.

Introduction and Motivation

Signal Temporal Logic (STL) inference provides interpretable formulas for time series classification, but accurate predictions do not necessarily ensure that the inferred formulas are satisfied consistently with the true class Abstract— Signal Temporal Logic (STL) inference provides interpretable formulas for time series classification. For multiclass classification, however, accurate predictions do not necessarily ensure that the inferred formulas are satisfied consistently with the true class. We introduce the positive satisfaction rate and codeword satisfaction rate to quantify satisfaction of required formulas and consistency with the complete class codeword, respectively

Existing multiclass STL models use margin based objectives to separate class dependent robustness patterns Tight STL inference instead seeks formulas that remain satisfied while approaching the satisfaction boundary These objectives concern different properties, where classification depends on the relative ordering of class decoding losses, whereas codeword satisfaction requires the robustness signs to agree with every entry of the true class codeword

Methodology Overview

The method proceeds in three steps: Discriminative learning trains formulas whose signs follow the codewords and whose decoding losses rank the true class first Refinement then keeps the formula structure and moves only the thresholds, pulling satisfied required formulas toward a small positive robustness while satisfaction, ranking, and anchor penalties limit the loss of satisfaction, class separation, and threshold drift The selected model is finally calibrated class by class using its decoding loss as the nonconformity score

Discriminative Multiclass STL Learning

The discriminative objective is defined as Ldisc = Lpos + λnegLneg − δm + λrankLrank(θ) The codewise losses are defined as Lpos = 1/P X (i,k)∈P [mk − Reθ,k(xi)]+, and Lneg = 1/N X (i,k)∈N [mk + Reθ,k(xi)]+ The ranking penalty Lrank(θ) encourages the true class to lead its closest competitor by at least µ

Tightness Refinement

Refinement fixes the selected logical structure and temporal intervals and adjusts only the predicate thresholds b ∈ Rnb The tightening term applies the TeLEx peak function Tβ(r) = 1/r + e−βr − e−r to the positive outputs of formulas required by the true class A separate satisfaction penalty Lsign penalizes existing violations and positive outputs below the satisfaction margin γ The refinement objective is Lref = ηtightLtight + ηsignLsign + ηrankLrank(b) + ηanchorLanchor

Class Conditional Conformal Calibration

The resulting prediction set is Cccα (x) = [c: dθ⋆ (x, c) ≤ qbc] where s(x, c) = dθ⋆ (x, c) as the nonconformity score This connection also explains when a class threshold can be zero Under exchangeability within each class, calibration provides the coverage guarantee stated in Sec II

Case Study Results

Experiments on naval, synthetic, and CALVIN trajectories demonstrate discrepancies between classification accuracy and formula satisfaction For the naval dataset, refinement tightens the formulas substantially on this dataset Class conditional calibration raises the lowest class coverage from 0.750 to 0.870, as Fig. 2 shows

Conclusion

Multiclass decoding can remain accurate even when the robustness signs fail to realize the intended class codeword PSR and CSR make this discrepancy explicit, while the refinement procedure exposes a controllable tradeoff between moving satisfied outputs toward the STL boundary and limiting losses in satisfaction and class separation Across the three datasets, the experiments show that the magnitude of attainable tightening is data dependent, while the distinction between decoding accuracy and specification satisfaction persists across all three datasets

--- Page 1 ---

Class-Conditional Conformal Prediction for Signal Temporal Logic Inference with Many Classes Yixuan Wang, Danyang Li, Matthew Cleaveland, Roberto Tron, Mingyu Cai are with Mechanical Engineering, University of California, Riverside, CA 92521, USA. Abstract— Signal Temporal Logic (STL) inference provides interpretable formulas for time series classification. For multiclass classification, however, accurate predictions do not necessarily ensure that the inferred formulas are satisfied consistently with the true class. We introduce the positive satisfaction rate and codeword satisfaction rate to quantify satisfaction of required formulas and consistency with the complete class codeword, respectively We develop a two-stage procedure that first learns discriminative formulas using a margin-based objective and then refines predicate thresholds toward a small positive robustness target while fixing the formula structure and temporal intervals Satisfaction and ranking penalties discourage violations of required formulas and loss of class separation The decoding loss is used as a nonconformity score and calibrated separately for each class, providing finite-sample class-conditional coverage under intra-class exchangeability Experiments on naval, synthetic, and robot manipulation trajectories demonstrate discrepancies between classification accuracy and formula satisfaction

Improvements for AI systems

The system can achieve finite-sample class-conditional coverage under intra-class exchangeability by using class-specific thresholds derived from a nonconformity score calibrated separately for each class, mitigating undercoverage that can be hidden by pooled calibration.
The AI system can achieve tighter temporal specifications than accuracy alone suggests by utilizing the tightness refinement stage, which adjusts predicate thresholds toward a small positive robustness target while fixing the formula structure and temporal intervals.
The system can explicitly quantify the gap between classification performance and logical specification adherence by reporting on positive satisfaction rate (PSR) and codeword satisfaction rate (CSR), which reveal how much accuracy alone fails to capture regarding formula requirements.
The AI system can perform robust, uncertainty-aware predictions by using a nonconformity score derived from the decoding loss, where its zero level corresponds to codeword consistency, allowing for class-conditional prediction sets that guarantee the inclusion of the true label under withinclass exchangeability.