Uncertainty Quantification for LLM Function-Calling
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Uncertainty Quantification for LLM Function-Calling".
Jane: Here's the gist of the paper. Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks, and this work presents what is, to our knowledge, the first evaluation of Uncertainty Quantification (UQ) methods for LLM Function-Calling (FC) <ref:2604.22985#pg2> While multi-sample UQ methods show strong performance for natural language Q&A tasks, they offer no clear advantage over simple single-sample UQ methods in the FC setting <ref:2604.22985#pg4> The particularities of FC outputs can be leveraged to improve existing UQ methods by tailoring them to the specific characteristics of these outputs <ref:2604.22985#pg2> Benchmarking and Methodology The authors build the first benchmark for UQ methods in Function-Calling settings based on the Berkeley Function Calling Leaderboard dataset (BFCL) <ref:2604.22985#pg4> This benchmark evaluates various UQ approaches by assessing how well their confidence estimates predict the correctness of function calls outputted by LLMs <ref:2604.22985#pg4> They reimplement several UQ methods, ranging from simple single-sample approaches based on model logits to more sophisticated multi-sample approaches that incorporate semantic equivalence between samples <ref:2604.22985#pg4> The evaluation is quantified using the Area Under the Receiver Operating Characteristic (AUROC) metric, which measures how well uncertainty estimates discriminate correct from incorrect function calls <ref:2604.22985#pg4> Performance Comparison The findings indicate that in the FC setting, unlike in natural language Q&A settings, more involved multi-sample methods do not outperform simple single-sample UQ methods <ref:2604.22985#pg4> Specifically, Semantic Entropy (SE) does not outperform Greedy Negative Log-Likelihood (G-NLL) as the best single-sample method <ref:2604.22985#pg4> The authors then investigate improving existing UQ methods by adapting them to FC output characteristics <ref:2604.22985#pg2> They adapt the clustering strategy in Semantic Entropy by using Abstract Syntax Trees (AST) to cluster FC outputs, and they adjust G-NLL and other single-sample UQ methods by selecting only semantically meaningful tokens when calculating UQ scores <ref:2604.22985#pg2> Adaptation of Single-Sample Methods The authors define Semantically Meaningful Tokens (SMT) as those corresponding to positions where the LLM has to decide between tokens that likely lead to either correct or incorrect outputs <ref:2604.22985#pg4> They define these tokens based on analyzing FC outputs, identifying positions related to deciding if functions are called, if the right function is called, if the right parameter is used, and if the right parameter value is chosen <ref:2604.22985#pg4> Implementing SMT Extraction involves a rule-based approach that classifies tokens based on their role in forming a correct function call output <ref:2604.22985#pg4> This leads to an adaptation of single-sample methods referred to as Semantically Meaningful Tokens (SMT) <ref:2604.22985#pg4> Overall, the work demonstrates that one can leverage FC output particularities to improve over existing UQ methods for LLMs, both for single- and for multi-sample UQ methods <ref:2604.22985#pg4> Specifically, they adapt the semantic-entailment method of multi-sample UQ methods by using Abstract Syntax Trees <ref:2604.22985#pg2> For single-sample UQ methods, they use only semantically meaningful tokens when calculating their UQ scores <ref:2604.22985#pg4> Limitations include the difficulty in comparing results on frontier models and the need for analysis in more challenging settings like multi-turn or agent settings <ref:2604.22985#pg4> The computational cost of multi-sample methods remains a barrier to surpassing single-sample methods <ref:2604.22985#pg4> Compute Infrastructure LLM inference for the BFCL tasks was performed using vLLM on a single node with 4 A100 GPUs, taking roughly 40 minutes in total to generate the greedy samples across all task datasets and models <ref:2604.22985#pg4> Generating 10 high-temperature samples per request required 3 hours in total for all task datasets and models <ref:2604.22985#pg4> Computation of all UQ scores for any single model and any (individual or combined) task dataset took less than a minute on a 2021 Apple MacBook Pro with M1 Max chip and 64GB RAM <ref:2604.22985#pg4> Risk Coverage Analysis The risk-coverage curves illustrate what the AUROC values correspond to in terms of potential accuracy gains thanks to UQ-based abstention <ref:2604.22985#pg4> For example, not executing a function call for the 30% of requests with highest G-NLLSMT UQ scores increases accuracies from < 92% to 96% for Simple tasks and from 75% to 85% for All Combined tasks <ref:2604.22985#pg4> The gains are similar across task combinations, reaching approximately +10% once the irrelevance task is added <ref:2604.22985#pg4> Calibration Insights The smooth Expected Calibration Error (smoothECE) shows that AVG achieves the worst calibration among all methods, being largely overconfident with confidence values higher than the accuracy <ref:2604.22985#pg4> Ptrue and G-NLL are underconfident, while MAX achieves the best calibration <ref:2604.22985#pg4> This illustrates that ECE does not necessarily correlate with AUROC <ref:2604.22985#pg4> All multi-sample entropy-based methods do not output probabilities, so their calibration cannot be evaluated directly AST Decoding Errors and Effective Sample Sizes On the All Combined split, 3.4% of the outputs are AST decoding errors averaged across 8 models <ref:2604.22985#pg4> The Spearman rank correlation between rankings when including AST decoding errors versus excluding them is 0.94, indicating that non-decodable function calls do not pose a risk from the perspective of preventing potentially harmful function calls <ref:2604.22985#pg4> Table 5 shows the effective sample size per model on the All Combined split Sensitivity to Hyperparameters Table 7 shows that increasing J to 30 can improve AUROC by around 0.02–0.
Title and authors: Tom: So we're looking at this paper called "Uncertainty Quantification for LLM Function-Calling". It tackles a big problem because LLMs are increasingly doing things on their own, and when they use tools or call functions—which is function calling—we need to know if they are right before they actually execute something.
Jane: Right. So, the core idea here is that these function calls can have serious consequences, like messing up data or losing money if the LLM calls a function incorrectly, and this paper presents what they call the first evaluation of uncertainty quantification methods for function calling.
Lu: Exactly. They are looking at how to quantify the model's confidence in a tool-use decision so we can stop it from making mistakes before it happens, which is critical when these actions can be irreversible <ref:2604.22985#pg1>.
Meng: From a production standpoint, that makes sense because if an LLM decides to delete something based on a faulty function call, you need a way to check that confidence score first <ref:2604.22985#pg3>.
Lalam: I’ve reviewed the summary of "Uncertainty Quantification for LLM Function-Calling", and it boils down to this: they look at how different uncertainty methods perform when the LLM is trying to call a function, and they found that while some multi-sample methods do well for normal question answering, they don't show an obvious advantage over simpler single-sample methods in this specific function calling setting <ref:2604.22985#pg4>.
Tom: So the paper benchmarks several approaches—ranging from simple methods based on the model's raw scores to more complex multi-sample methods that look at how similar different samples are semantically <ref:2604.22985#pg4>.
Jane: That's interesting because the authors suggest that they can actually improve these existing uncertainty methods by tailoring them specifically to what function calls look like, which is a key point they make in page two <ref:2604.22985#pg2>.
Lu: They explore adapting things, and for example, they take the clustering strategy used in Semantic Entropy and change it to use Abstract Syntax Trees or ASTs to group the function calls together <ref:2604.22985#pg3>.
Meng: AST matching sounds like a way to handle the structure of the function call better than just comparing text strings, which is what I'd expect in structured output like this <ref:2604.22985#pg3>.
Lalam: And for the single-sample methods, they adapt them by only looking at tokens that are semantically meaningful when calculating the uncertainty scores <ref:2604.22985#pg4>.
Tom: So this means they're not just throwing more complexity at it; they are trying to make the existing tools smarter by focusing their attention on the parts of the call that actually matter for correctness, like whether a specific function is chosen or if a parameter is correct <ref:2604.22985#pg4>.
Title and authors: Jane: That leads into how they define these semantically meaningful tokens, which they call SMTs. Basically, it's about identifying the spots in the function call where the LLM has to make a tough choice between different tokens that could lead to a right or wrong outcome <ref:2604.22985#pg4>.
Lu: They use a rule-based approach for this extraction, classifying tokens based on their role in forming a correct function call output, which is how they create these SMTs <ref:2604.22985#pg4>.
Meng: Rule-based token classification sounds like it adds some complexity to the pipeline, but if it isolates the high-leverage parts of the decision, it could be worth the effort <ref:2604.22985#pg4>.
Lalam: And they then apply this SMT concept to improve single-sample methods by only using those tokens when calculating their uncertainty scores <ref:2604.22985#pg4>.
Tom: So, one big thing here is that they show you can get better results for both the simple and multi-sample methods just by tweaking them to understand the specific characteristics of function calls instead of using a general approach <ref:2604.22985#pg4>.
Jane: That's the main implication, Tom. They demonstrate that because function calling has these unique structural properties, we can make existing uncertainty tools work better for this task <ref:2604.22985#pg1>.
Lu: It’s a practical step toward making uncertainty quantification more useful in real-world autonomous agents, moving beyond just accuracy metrics to true risk assessment <ref:2604.22985#pg3>.
Meng: I worry about the computational cost of these new methods. If we have to run AST analysis or complex token selection every time, it adds latency that we can't afford at scale <ref:2604.22985#pg4>.
Tom: That’s a valid point, Meng. But they did show some speed tests on the infrastructure side; for instance, generating the greedy samples across all datasets took about forty minutes on their setup with four A100 GPUs <ref:2604.22985#pg4>.
Jane: They also showed how this uncertainty translates into potential risk reduction. For example, by not executing a function call when its SMT uncertainty score is high, they saw accuracy jump from ninety-two percent to ninety-six percent on simple tasks <ref:2604.22985#pg4>.
Lu: That jump in accuracy shows that the abstention based on their UQ scores actually prevents potentially harmful function calls, which is what we need for safety <ref:2604.22985#pg3>.
Lalam: And they also looked at the calibration, showing that methods like MAX uncertainty can achieve the best calibration among all the ones they tested, meaning their confidence scores are more honest about how likely they are to be correct <ref:2604.22985#pg4>.
Tom: So, to wrap up on this paper, "Uncertainty Quantification for LLM Function-Calling", the main thing is that we can leverage the specific structure of function calls—using ASTs and SMTs—to make uncertainty quantification methods work better for tool use <ref:2604.22985#pg1>.
Jane: And they proved that adapting these techniques allows them to improve both the single-sample and multi-sample methods we already have, which is a useful direction for everyone working on this area <ref:2604.22985#pg1>.
Title and authors: Lu: The paper highlights how these adaptations, specifically using ASTs for clustering and SMTs for scoring, lead to tangible improvements in performance metrics like the AUROC <ref:2604.22985#pg4>.
Meng: But they also pointed out a limitation: comparing results across different frontier models is hard, and they need more work in settings that are more complex than just single function calls, like multi-turn conversations or agent workflows <ref:2604.22985#pg4>.
Tom: And one of the other things they noted is that the computational cost of multi-sample methods still makes them slower than the simpler single-sample approaches, unless you use these new adaptations <ref:2604.22985#pg4>.
Jane: So, while there are clear improvements in accuracy and calibration when we tailor the methods to function calling outputs, we still have to deal with the challenge of making sure these methods scale efficiently for real deployment <ref:2604.22985#pg4>.
Lu: It’s a solid foundation because it shows a path forward for uncertainty quantification in this specific area, moving from general Q andA performance to specialized, safer function calling decisions <ref:2604.22985#pg3>.
Meng: I'll keep an eye on the computational cost aspect. If we can deploy these adapted SMT methods without a massive overhead, that’s where this research has real value for production systems <ref:2604.22985#pg4>.
Tom: So, in summary, the work on "Uncertainty Quantification for LLM Function-Calling" shows we can use the specific details of function calls—like their syntax and argument selection—to refine how we measure model confidence <ref:2604.22985#pg1>.
Jane: It’s about moving past just seeing if a model is right or wrong in general, and instead learning how to predict which specific action it might take incorrectly <ref:2604.22985#pg1>.
Lu: This work gives us concrete methods—using AST clustering and SMT tokens—to make multi-sample UQ methods more effective for function calling tasks <ref:2604.22985#pg3>.
Meng: The practical implication is that if we can get the cost down, these tailored uncertainty scores give us a much safer way to deploy LLMs that use tools <ref:2604.22985#pg4>.
Tom: We’ll keep an eye on the future work they mention, especially how these methods fare when we move into more complex agent settings or multi-turn scenarios <ref:2604.22985#pg4>.
Jane: That sounds like a natural next step for any system that’s actually going to be used autonomously in the real world <ref:2604.22985#pg3>.
Lu: It really sets the stage for understanding how robust these systems are when they are operating under pressure, which is where we need to focus our research next <ref:2604.22985#pg4>.
Meng: Yeah, so the paper on "Uncertainty Quantification for LLM Function-Calling" provides a clear framework for making tool use safer by focusing on the structure of those calls <ref:2604.22985#pg1>.
Tom: It’s a solid piece of work because it shows that even in this complex area, we can get more reliable confidence estimates by adapting the methods to the task at hand <ref:2604.22985#pg4>.
The paper's summary: Tom: So, we're talking about how these papers—and this one specifically—are looking at uncertainty quantification for function calling, and it boils down to this: they're checking if the ways we measure a model’s confidence when it tries to call a tool are actually good enough for real use.
Jane: Exactly. Basically, the authors look at how different methods work when an AI is trying to use a function, and they found that while some complex methods do well on regular question answering, they don't show any clear advantage over simpler single-sample methods in this specific function calling setting.
Lu: Right. They’re saying you don't always need the most complicated multi-sample methods when dealing with these structured outputs from an LLM.
Meng: So what does that mean for us shipping models? If a more complex setup isn't better, we can probably stick to simpler, faster uncertainty checks for deployment.
Tom: Actually, they go further than just saying one is better than another. They show you can actually improve these existing methods by tailoring them specifically to the way function calls look.
Jane: That’s the interesting part because they propose two main ways to do that. First, they adapt the clustering strategy in a method called Semantic Entropy by using Abstract Syntax Trees or ASTs to group those function calls together instead of just matching strings.
Lu: Using ASTs means you're looking at the actual structure of the code being generated, treating it like a dictionary where arguments are interchangeable, which is much better for structured outputs than simple string matching.
Tom: And for the single-sample methods, they adapt those by only calculating uncertainty scores based on semantically meaningful tokens. They call these Semantically Meaningful Tokens or SMTs.
Jane: What does that mean practically? It means instead of looking at every single word the LLM spits out, they focus only on the parts where the model has to make a critical decision—like which function to pick, or which parameter value is right.
Lu: They define these SMTs using a rule-based approach that classifies tokens based on their role in forming a correct function call output, so it's not just guesswork.
Meng: That sounds like it adds complexity to the pre-processing step, but if it isolates the high-leverage parts of the decision, that could be worth the effort for safety.
Tom: They test this out and show that using these adaptations—like AST clustering for multi-sample methods and focusing on SMTs for single-sample methods—actually leads to better performance scores, like AUROC.
Jane: And they also looked at the results in terms of risk coverage. For example, by not executing a function call when its SMT uncertainty score is high, they saw accuracy jump from ninety-two percent to ninety-six percent on simple tasks.
Lu: That jump shows that this abstention based on their specific uncertainty scores actually prevents potentially harmful function calls in practice.
Tom: They also checked the calibration, and they found that some methods, like MAX uncertainty, achieve the best calibration among all the ones they tested. This means their confidence scores are more honest about how likely they are to be correct.
Jane: It’s a big deal because it shows that we can move past just checking if a model is right or wrong in general, and instead learn how to predict which specific action it might take incorrectly, with more accurate confidence scores.
Lu: But they did flag some limitations. They mentioned that comparing the results across different frontier models is really hard and they need to do more work in settings that are way more complex than just a single function call, like multi-turn conversations or full agent workflows.
Meng: That makes sense for me because running these AST analyses on every single request could add a lot of latency at scale.
Tom: Yeah, the computational cost of those multi-sample methods is still a barrier to surpassing the simpler single-sample approaches unless you use these new adaptations, so that's where we need to focus our engineering efforts.
Jane: So, this paper gives us a clear path forward for uncertainty quantification in this specific area by focusing on the structure of function calls and how we measure model confidence more accurately.
The paper's improvements: Tom: So, we're looking at how they actually suggested making these uncertainty quantification methods better in the paper, and it’s really about taking those ideas and putting them into a working system for function calling.
Jane: Right. The core of their suggestion is to make things adaptive. They want the system to dynamically choose between different tools, not just stick to one fixed method no matter what.
Lu: Specifically, they suggest adapting single-sample methods like Greedy Negative Log-Likelihood, or G-NLL, by making them switch over to tailored single-sample methods like Semantically Meaningful Tokens or SMTs when the model is making a call.
Meng: So instead of running the same uncertainty check every time, we could analyze the function call output first to see if it’s complex enough to warrant using a more detailed SMT analysis.
Tom: Exactly, and they back this up by showing that adapting methods like Semantic Entropy also works well when you use Abstract Syntax Trees for clustering those samples together.
Jane: That’s interesting because it means the system gets smarter about what makes a function call unique, rather than just treating every output the same way.
Lu: The implication there is that you can use these techniques to improve both single-sample and multi-sample uncertainty methods without necessarily adding massive computational overhead for every task.
Tom: They also pointed out that when they used the MAX uncertainty method alongside those SMT adaptations, it achieved the best calibration among all the methods they tested, which is a nice bonus because it means the confidence scores are more accurate.
Jane: That’s important because when you’re actually building something that uses these AI tools, you need to trust that confidence score. If MAX is better calibrated, then we can make more reliable decisions about whether to run a function or not.
Lu: It moves the goal from just getting a high AUROC score on a test set to having an uncertainty estimate that actually reflects the true probability of correctness for the specific action being taken.
Meng: I still have my reservations about deployment cost, though. If this adaptive switching requires constant deep analysis of every output, we need to make sure the performance gain is worth that added latency when it’s running live.
Tom: They did touch on that too, but they showed that for the single-sample adaptation, using only semantically meaningful tokens keeps the calculation focused and efficient.
Jane: So what this changes for someone just listening? It means we can expect future AI systems to be better at telling us *why* they are unsure about a tool call, not just *how* unsure they are.
Lu: That’s the big picture: it’s about giving these autonomous agents a way to communicate their own internal decision-making uncertainty in a way that is actually grounded in the structure of the task.
Conclusion: Tom: So we're wrapping up this look at "Uncertainty Quantification for LLM Function-Calling", and the main point is that we can take existing tools and make them much smarter by focusing on how function calls are actually structured.
Jane: Right, so it’s not just about building a bigger model; it’s about learning the specific language of tool use so we can measure risk more reliably.
Lu: The paper showed that by using Abstract Syntax Trees for clustering and Semantically Meaningful Tokens for single-sample scoring, we get concrete improvements on metrics like AUROC.
Meng: I still have my reservations about deployment cost though. If this adaptive switching requires constant deep analysis of every output, we need to make sure the performance gain is worth that added latency when it’s running live.
Tom: Yeah, the computational cost of those multi-sample methods is still a barrier to surpassing the simpler single-sample approaches unless you use these new adaptations.
Jane: And they did flag that comparing results across different frontier models is really hard, and they need to do more work in settings that are way more complex than just a single function call, like multi-turn conversations or full agent workflows.
Lu: That’s the next big challenge, moving from static function calls to dynamic agent actions where the structure changes constantly.
Lalam: I’m confident that these adaptations provide a solid foundation for making tool use safer by focusing on the syntax and parameter selection of those calls.
Tom: So in summary, "Uncertainty Quantification for LLM Function-Calling" gives us concrete methods—using ASTs and SMTs—to make multi-sample UQ methods more effective for function calling tasks.
Jane: It’s about moving past just seeing if a model is right or wrong in general, and instead learning how to predict which specific action it might take incorrectly, with more accurate confidence scores.
Lu: This work gives us a clear path forward for uncertainty quantification in this specific area by focusing on the structure of function calls and how we measure model confidence more accurately.
Meng: I'll keep an eye on the future work they mention, especially how these methods fare when we move into more complex agent settings or multi-turn scenarios.
Tom: So that’s it for this paper. It shows that even in this complex area, we can get more reliable confidence estimates by adapting the methods to the task at hand.
Jane: It’s about moving past just seeing if a model is right or wrong in general, and instead learning how to predict which specific action it might take incorrectly.
Lu: The implication there is that these tailored uncertainty scores provide a more robust way to assess risk when an AI is using tools.
Zihuiwen Ye, Lukas Aichberger, Michael Kirchhof, Sinead Williamson, Luca Zappella, Yarin Gal
University of Oxford · Apple
cs.CL
Submitted: 2026-04-24
Updated: 2026-10-30
Journal ref: TMLR 2026
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 95/100
The gist: Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks, and this work presents what is, to our knowledge…
Key concepts
- Function-Calling (FC)
- This is when a Large Language Model (LLM) is asked to generate specific code or calls to external functions, rather than just answering a question. It requires the model to output structured data that represents an action, like calling a specific API function with correct parameters.
- Uncertainty Quantification (UQ)
- UQ methods are techniques used to measure how confident an LLM is in its own prediction. In this context, they estimate the probability that the generated function call is correct or incorrect, helping users decide whether to trust the model's output.
- Semantically Meaningful Tokens (SMT)
- SMT refers to specific words or tokens within an LLM's output that are crucial for determining correctness. These are tokens where the model must choose between options that lead to a right or wrong function call, such as deciding which function name to use.
- Abstract Syntax Trees (AST)
- ASTs are a tree-like structure used in programming to represent the structure of code. When applied here, ASTs help cluster different function calls based on the actual structure of their arguments, allowing multi-sample UQ methods to better understand semantic similarity.
Terminology
Summary
Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks, and this work presents what is, to our knowledge, the first evaluation of Uncertainty Quantification (UQ) methods for LLM Function-Calling (FC) <ref:2604. p. 0 Uncertainty Quantification for LLM Function-Calling
Zihuiwen Ye∗
University of Oxford
Lukas Aichberger∗
University of Oxford
Michael Kirchhof
Apple
Sinead Williamson
Apple
Luca Zappella
Apple
Yarin Gal
University of Oxford
Arno Blaas+
Apple
Adam Goliński+
Apple
Abstract
Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world
tasks. A key ingredient for this is the LLM Function-Calling paradigm, a widely used
approach for equipping LLMs with tool-use capabilities. In this work, we present what is, to our knowledge, the
first evaluation of UQ methods for LLM Function-Calling (FC). While multi-sample UQ
methods, such as Semantic Entropy, show strong performance for natural language Q&A
tasks, we find that in the FC setting, it offers no clear advantage over simple single-sample
UQ methods. While multi-sample UQ
methods, such as Semantic Entropy, show strong performance for natural language Q&A
tasks, we find that in the FC setting, it offers no clear advantage over simple single-sample
UQ methods. Additionally, we find that the particularities of FC outputs can be leveraged to
improve the performance of existing UQ methods in this setting.Where the paper says it
Where the paper says it
Where the paper says it
Benchmarking and Methodology
The authors build the first benchmark for UQ methods in Function-Calling settings based on the Berkeley Function Calling Leaderboard dataset (BFCL) <ref:2604.22985#pg4> This benchmark evaluates various UQ approaches by assessing how well their confidence estimates predict the correctness of function calls outputted by LLMs <ref:2604. p. 1 , 2024), which provides a collection of
FC examples, mostly in Python. Our benchmark evaluates various UQ approaches based on how well their
confidence estimates predict the correctness of function calls outputted by LLMs. Our benchmark evaluates various UQ approaches based on how well their
confidence estimates predict the correctness of function calls outputted by LLMs. We reimplement several
UQ methods, ranging from simple single-sample approaches based on model logits to more sophisticated
multi-sample approaches that incorporate semantic equivalence between samples, and test how effectively
they discriminate correct from incorrect function calls in a variety of setups. The UQ-performance is then quantified as the performance of this
classifier. The most appropriate metric to use for this is the area under the receiver-operation characteristic
(AUROC) (Fawcett, 2006).Where the paper says it
Where the paper says it
Where the paper says it
Performance Comparison
The findings indicate that in the FC setting, unlike in natural language Q&A settings, more involved multi-sample methods do not outperform simple single-sample UQ methods <ref:2604.22985#pg4> Specifically, Semantic Entropy (SE) does not outperform Greedy Negative Log-Likelihood (G-NLL) as the best single-sample method <ref:2604. p. 1 We reimplement several
UQ methods, ranging from simple single-sample approaches based on model logits to more sophisticated
multi-sample approaches that incorporate semantic equivalence between samples, and test how effectively
they discriminate correct from incorrect function calls in a variety of setups. Our findings indicate that in the FC setting, unlike in natural language Q&A settings, the more involved
multi-sample methods, such as Semantic Entropy (SE), do not outperform simple single-sample UQ methods,
such as the Greedy Negative Log-Likelihood (G-NLL). We then investigate if the performance of existing
UQ methods in the FC setting can be increased by tailoring them to the characteristics of FC outputs. By adapting the clustering strategy in Semantic Entropy and the token selection in G-NLL, we achieve
performance improvements for both multi-sample and single-sample UQ methods. We analyze the properties of FC outputs and use them to devise variants of existing UQ methods that
are tailored to the FC setting and improve their performance in this setting. Namely, we adapt the
semantic-entailment method of SE to the FC setting by using Abstract Syntax Trees (AST) to cluster FC
outputs, and adjust G-NLL and other single-sample UQ methods to the FC setting by defining semantically
meaningful tokens and including only those in calculating UQ scores.Where the paper says it
Where the paper says it
Where the paper says it
Adaptation of Single-Sample Methods
The authors define Semantically Meaningful Tokens (SMT) as those corresponding to positions where the LLM has to decide between tokens that likely lead to either correct or incorrect outputs <ref:2604.22985#pg4> They define these tokens based on analyzing FC outputs, identifying positions related to deciding if functions are called, if the right function is called, if the right parameter is used, and if the right parameter value is chosen <ref:2604. p. 8 Tokens deciding if the right parameter value is chosen ( 1, 8, 0, 0, 1, 9, 0, 0, War, ,, E, conomy, The
Thinker, August, e, Rod, in, The, Kiss, August, e, Rod, in)
5. Tokens deciding if the right number of functions is called (]), ,), )])
6. Implementing SMT Extraction Once defined, there are many options to extract the semantically
meaningful tokens from a given LLM function-call output. We opt for a simple, rule-based approach that is
both computationally efficient and easy to implement. Once the semantically meaningful tokens are defined and extracted, we aggregate the corresponding logprobabilities in the same way as for the whole token sequence (MAX, G-NLL, AVG). This leads to an
adaptation of single-sample methods to the FC setting we refer to as Semantically Meaningful Tokens (SMT).Where the paper says it
Where the paper says it
Where the paper says it
Conclusion and Future Work
Overall, the work demonstrates that one can leverage FC output particularities to improve over existing UQ methods for LLMs, both for single- and for multi-sample UQ methods <ref:2604.22985#pg4> Specifically, they adapt the semantic-entailment method of multi-sample UQ methods by using Abstract Syntax Trees <ref:2604. p. 9 We further show that the particularities of the FC setting can be utilized to improve existing
UQ methods. Specifically, we adapt the semantic-entailment method of multi-sample UQ methods by using
Abstract Syntax Trees. We analyze the properties of FC outputs and use them to devise variants of existing UQ methods that
are tailored to the FC setting and improve their performance in this setting. Namely, we adapt the
semantic-entailment method of SE to the FC setting by using Abstract Syntax Trees (AST) to cluster FC
outputs, and adjust G-NLL and other single-sample UQ methods to the FC setting by defining semantically
meaningful tokens and including only those in calculating UQ scores. Also, we believe that our findings for smaller and medium-sized models are still very relevant,
especially in the context of on-device function calling, which would need to rely on such smaller models. Second, it would be interesting to conduct this analysis also for more challenging function-calling settings,
such as in multi-turn or agent settings. In this work, we present what is, to our knowledge, the
first evaluation of UQ methods for LLM Function-Calling (FC). While multi-sample UQ
methods, such as Semantic Entropy, show strong performance for natural language Q&A
tasks, we find that in the FC setting, it offers no clear advantage over simple single-sample
UQ methods.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Compute Infrastructure
LLM inference for the BFCL tasks was performed using vLLM on a single node with 4 A100 GPUs, taking roughly 40 minutes in total to generate the greedy samples across all task datasets and models <ref:2604.22985#pg4> Generating 10 high-temperature samples per request required 3 hours in total for all task datasets and models <ref:2604.22985#pg4> Computation of all UQ scores for any single model and any (individual or combined) task dataset took less than a minute on a 2021 Apple MacBook Pro with M1 Max chip and 64GB RAM <ref:2604.22985#pg4>
Risk Coverage Analysis
The risk-coverage curves illustrate what the AUROC values correspond to in terms of potential accuracy gains thanks to UQ-based abstention <ref:2604.22985#pg4> For example, not executing a function call for the 30% of requests with highest G-NLLSMT UQ scores increases accuracies from < 92% to 96% for Simple tasks and from 75% to 85% for All Combined tasks <ref:2604.22985#pg4> The gains are similar across task combinations, reaching approximately +10% once the irrelevance task is added <ref:2604. p. 3 032
All Combined + Irrelevance 0. 024
Table 1: Results: Mean AUROC ± mean standard error (mean over eight LLMs) for individual tasks, task
combinations, and unanswerable requests described in Sec.Where the paper says it
Calibration Insights
The smooth Expected Calibration Error (smoothECE) shows that AVG achieves the worst calibration among all methods, being largely overconfident with confidence values higher than the accuracy <ref:2604.22985#pg4> Ptrue and G-NLL are underconfident, while MAX achieves the best calibration <ref:2604. p. 5 Lastly, MAX achieves the best calibration. It is still slightly underconfident for most parts, but since by definition maxi
log pi ≥
P
i
log pi
, it is less
underconfident than G-NLL, especially in low-confidence regions. It is still slightly underconfident for most parts, but since by definition maxi
log pi ≥
P
i
log pi
, it is less
underconfident than G-NLL, especially in low-confidence regions. These insights complement the previous
AUROC results, and illustrate that ECE does not necessarily correlate with AUROC.Where the paper says it
Where the paper says it
AST Decoding Errors and Effective Sample Sizes
On the All Combined split, 3.4% of the outputs are AST decoding errors averaged across 8 models <ref:2604. p. 4 Throughout our analysis, we exclude the requests for which the greedy decoding results in an output that
could not be interpreted by the Python interpreter, i. , that result in an Abstract Syntax Tree (AST)
decoding error. These
errors are rare (on average 3. 4% of outputs on the All Combined split), and including them does not change
the relative ranking of UQ methods (Spearman correlation 0. This, in turn, can be used
to refuse to execute certain (potentially erroneous) function calls. This aspect, which is crucial from the
reliability perspective, to the best of our knowledge, so far has been neglected in tool-use evaluation.Where the paper says it
Where the paper says it
Where the paper says it
Sensitivity to Hyperparameters
Table 7 shows that increasing J to 30 can improve AUROC by around 0.02–0.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
- Adaptive Uncertainty Quantification for Function Calling (FC) Systems
The improved system can dynamically select between single-sample methods like Greedy Negative Log-Likelihood (G-NLL) and tailored single-sample methods like Semantically Meaningful Tokens (SMT). This adaptation is achieved by analyzing FC output properties, such as identifying semantically meaningful tokens, to calculate uncertainty scores only for those tokens.
- AST Clustering for Multi-Sample Uncertainty Estimation
Multi-sample UQ methods can be enhanced by replacing exact string matching with Abstract Syntax Tree (AST) matching when clustering function calls. This allows the system to treat function arguments as a dictionary that is permutation invariant, leading to better performance on structured FC outputs compared to simple EXM clustering.
- Improved Confidence Calibration for Decision Making
The system will utilize the MAX uncertainty method alongside SMT adaptation to improve confidence calibration, as evidenced by its best calibration
result in Figure 10. This ensures that the output confidence values more accurately reflect the true probability of correctness, which is crucial for reliable decision-making when deciding whether to execute a function call.
Abstract
Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks. A key ingredient for this is the LLM Function-Calling paradigm, a widely used approach for equipping LLMs with tool-use capabilities. However, an LLM calling functions incorrectly can have severe implications, especially when their effects are irreversible, e.g., transferring money or deleting data. Hence, it is of paramount importance to consider the LLM's confidence that a function call solves the task correctly prior to executing it. Uncertainty Quantification (UQ) methods can be used to quantify this confidence and prevent potentially incorrect function calls. In this work, we present what is, to our knowledge, the first evaluation of UQ methods for LLM Function-Calling (FC). While multi-sample UQ methods, such as Semantic Entropy, show strong performance for natural language Q&A tasks, we find that in the FC setting the entropy-based multi-sample methods we evaluate offer no clear advantage over simple single-sample UQ methods. Additionally, we find that the particularities of FC outputs can be leveraged to improve the performance of existing UQ methods in this setting. Specifically, multi-sample UQ methods benefit from clustering FC outputs based on their abstract syntax tree parsing, while for single-sample UQ methods, restricting the logit-based score to semantically meaningful tokens leaves discrimination essentially unchanged but improves calibration and, by smaller margins, the utility of the score under abstention.
Sources
- GPT-4 Technical Report
- MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agents
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing
- Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models
- Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification
- Gemini: A Family of Highly Capable Multimodal Models
- The Llama 3 Herd of Models
- Mixtral of Experts
- Language Models (Mostly) Know What They Know
- Pretrained Visual Uncertainties
- Robust Decision Making with Partially Calibrated Forecasts
- Augmented Language Models: a Survey
- Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data