ThinkSum: Probabilistic reasoning over sets using large language models
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "ThinkSum: Probabilistic reasoning over sets using large language models".
Title and authors: Tom: So, we're looking at this paper called ThinkSum, and what it’s doing is trying to fix a problem where large language models struggle with things that need multiple steps of logic or reasoning over several facts at once.
Jane: Right, so the title itself says "Probabilistic reasoning over sets using large language models," and that sounds like it's moving away from just asking the model one long question at a time.
Lu: Exactly. The core idea here is to structure the LLM's thinking into two distinct phases: first you get associations, and then you aggregate those associations probabilistically to get the final answer.
Meng: So, what does this actually mean for how we build these things? Is it just a fancy way of writing a prompt that works better?
Tom: No, it's different because the thinking part—the "Think" stage—is done by querying an LLM in parallel over several small phrases you extract from the prompt or maybe some other model call.
Jane: That sounds like breaking down a big problem into smaller pieces that you can check all at once before you put them back together.
Lu: Precisely, and the second part, the "Sum" stage, is where you take all those individual results from the first stage and combine them using some probabilistic inference techniques to make the final prediction.
Meng: I’m wondering what kind of things they're calling these inference techniques. Are they just simple voting mechanisms? Because when I try to run stuff at scale, I always worry about the overhead of running too many parallel calls.
Tom: They use several methods, including majority or minority votes and ratio of likelihoods for aggregation, but they also do product aggregation, which is multiplying the LLM likelihoods for all the strings in a set.
Jane: That sounds like you’re taking all those different potential paths the model suggests and mathematically combining their probabilities to find the most likely final outcome.
Lu: And they've even looked at more advanced structures, like using latent variable models in that Sum stage to assume an underlying structure for those items and facts.
Meng: Latent variables sound ambitious when you think about deployment. How does that help us figure out what actually runs on a production system?
Tom: It helps by assuming there's a hidden cluster, so you select the facts or items that belong to the majority class, which can simplify the decision process.
Jane: So instead of checking every single possible combination, you’re leaning into an assumed underlying structure to guide your selection.
Meng: That makes sense if we think about reducing the search space when we have a million possible inputs. But what about prompt design sensitivity? Isn't that a big issue with current methods?
Tom: That's where ThinkSum really shines because the reasoning step is performed outside of the LLM’s direct generation process, which makes it less sensitive to things like accidental distractions in the prompt.
Title and authors: Jane: So if you accidentally put a weird word in your prompt that confuses the model, it doesn't derail this entire set-based inference structure as much?
Lu: It reduces vulnerability because they engineer that slow thinking process to make parallel calls for intermediate info before combining them, which isn't just letting the LLM do all the heavy lifting in one go.
Meng: So it’s more robust to prompt design, but what about the cost? I need to know if this scales reasonably when we have a whole set of objects or facts to reason over.
Tom: The computation cost scales with how many parallel calls you make, which depends on the number of Think prompts you use for each task and how many objects you need likelihood computations for at the Sum stage.
Jane: So it’s not free scaling; it costs more as the problem gets bigger in terms of objects. But they say it's generally faster than some chain-of-thought methods because those are sequential generation problems, whereas ThinkSum does parallel calls.
Lu: And for tasks like logical deduction, they translated the problems into sets of mathematical inequalities and then used product aggregation followed by mixture aggregation to determine the correct ordering.
Meng: That sounds like a solid way to handle complex constraints without having to force the LLM into generating a perfect step-by-step narrative. But what about those tasks where it’s not working well?
Tom: They tested it across thirteen difficult tasks from the BIG-bench suite, and they showed improvements over the state of the art using GPT family models on thirteen specific difficult ones.
Jane: And they found that for a task like logical deduction with five objects, ThinkSum got an accuracy of seventy-seven percent, which is better than what average humans get for that specific type of problem.
Meng: Seventy-seven percent is good, but I need to know how it holds up when we move to real traffic or complex video reasoning problems where the input isn't just text sets.
Tom: They also noted that in the ODD ONE OUT task, they maintained near-average human accuracy for models ranging from GPT two small to the largest GPT three model.
Jane: So it’s pretty consistent across different model sizes for those specific benchmark tasks, which is encouraging when you're trying to deploy something.
Lu: The difference between ThinkSum and traditional chain-of-thought methods, like the ones from Wei et al., is that the reasoning isn't reduced to a generation problem for the LLM anymore.
Meng: So you’re not just chaining text outputs together; you’re doing something different with those intermediate results before you hit the final answer.
Tom: Yeah, thinkSum performs classical probabilistic reasoning externally, which is what allows it to reduce vulnerability to prompt features like spurious patterns that might confuse the LLM.
Jane: That external reasoning step means the structure of your set-based approach is more important than making the LLM write a perfect little narrative for you.
Lu: And they’ve shown this can be combined with latent variable models to extract structured knowledge from the LLMs, which is really interesting for understanding what’s actually happening inside those large models.
Title and authors: Meng: That sounds like a way to pull out hidden knowledge without having to retrain the whole model just for that one set of facts. What about practical deployment costs again?
Tom: The cost scales with the number of parallel calls, which you determine based on how many Think prompts you use and how many objects you need likelihood computations for at the Sum stage.
Jane: So if you have a really massive set, that computational load will grow significantly because of those parallel calls. It’s not a cheap operation for very large inputs.
Lu: The authors also pointed out a limitation: as the number of objects or facts you're reasoning over increases, the computation cost will also rise.
Meng: That’s the caveat I was worried about; scaling up means scaling up the compute demand for this inference structure.
Tom: There’s another point, though, that when you are solving a brand new task, it requires human effort to select specific operations in each step for both Think and Sum.
Jane: So even if the framework is good at reasoning over sets, you still have to figure out the right sequence of operations for a novel problem.
Lu: Future work they suggest could focus on automating that prompt engineering procedure so you don't have to manually select every operation every time.
Meng: It sounds like they’re focused on making the process more automated or at least less manual for the user who has to set it up.
Tom: So, in short, ThinkSum is a two-stage approach that uses parallel LLM queries for association and then probabilistic aggregation for reasoning over sets.
Jane: It’s a way to use LLMs to handle complex logical deductions by treating the problem as a set of facts instead of just one long sequence.
Lu: It offers more interpretability because you can trace the results back through those explicit retrieval steps, which is useful when we want to understand *why* an AI made a certain choice.
Meng: For us in production, if this means less sensitivity to prompt formatting errors and more structured output, that could be a win for stability.
Tom: It really does sound like they're proposing a way to get more reliable reasoning from existing large models by controlling the inference step outside of the model itself.
Jane: So we’ve seen how this two-stage paradigm works and what it means for tackling problems that require reasoning over multiple objects or facts.
Lu: The paper "ThinkSum: Probabilistic reasoning over sets using large language models" proposes this structured approach to make LLMs more reliable when dealing with complex logical deductions.
Meng: It sounds like a promising direction, especially with the focus on probabilistic inference being done externally from the LLM's core generation.
Tom: We’ll keep an eye on how they extend this paradigm to newer models and how they handle those scaling costs as we move towards even larger architectures.
The paper's summary: Tom: So, basically, ThinkSum is this two-stage approach where you use an LLM to quickly grab associations from a set of phrases, and then you take all those results and combine them using probability math to get the final answer.
Jane: It’s trying to fix a problem where models struggle with reasoning over multiple facts or objects at the same time, which is what this paper is all about.
Tom: Right. They show that instead of just having the LLM generate one long chain of thought, you can do this structured thinking process externally and then perform the inference outside of the LLM itself.
Jane: And they found that because you're doing that probabilistic math outside of the model, it makes the whole system much more robust to how you write your prompt—it’s less sensitive to accidental distractions.
Tom: Exactly. They tested this on thirteen difficult tasks from a benchmark suite called BIG-bench, and they actually got better results than the state of the art using GPT family models on thirteen specific tough challenges.
Jane: For instance, in a task that involves logical deduction with five objects, ThinkSum achieved seventy-seven percent accuracy which beats the average human performance for that type of problem.
Tom: That’s a concrete number. And they found that this method outperformed traditional chain-of-thought prompting on all those tasks except for one specific variant, KNOWN UNKNOWNS with the davinci model.
Jane: So what does it mean practically? It suggests that we can get more reliable reasoning from existing large models just by adding a smart controller layer that handles the set reasoning and probabilistic inference.
Tom: The cost analysis shows that the computation scales with how many parallel calls you make, which depends on how many Think prompts you use and how many objects you need to run likelihood computations for at the Sum stage.
Jane: So it’s not free scaling; if you have a really big set of facts, the computation load goes up because of those parallel calls. But they also mentioned that when you're starting a brand new task, you still need human effort to select the right operations for both stages.
Tom: That’s the limitation they flagged. The authors say future work needs to focus on automating that process of selecting the right operations so it’s less manual for users.
Jane: Still, this whole structure—using parallel retrieval followed by structured probabilistic aggregation—gives us a way to see what's happening inside these models more clearly because you can trace the results back through those explicit steps.
Tom: It’s about treating the LLM like a retrieval and aggregation engine rather than just a pure generation machine for complex tasks.
Jane: And that structure, combined with latent variable models, lets them extract structured knowledge from the AI that they couldn't get before.
Tom: So, ThinkSum is proposing this two-stage paradigm—Think for associations, Sum for inference—to enhance reasoning over sets. But where does it leave us when we think about deploying this kind of structured thinking on a massive scale?
The paper's improvements: Tom: So, we're looking at how they actually make ThinkSum better than just using a standard chain of thought prompt to solve these set problems.
Jane: Right, so they aren't just relying on the LLM to write a long narrative anymore; they’ve introduced a two-stage process that does retrieval first, and then probabilistic inference second.
Tom: Exactly. The first stage is "Think," where the LLM queries several small phrases in parallel—maybe things like "Order inversion" or generating a list of facts about cats—to build up a set of candidate associations.
Jane: And then in the second stage, the "Sum," they take all those different results and use specific math techniques like mixture aggregation or product aggregation to pick the most likely final answer.
Tom: They also introduced this latent variable model approach for the Sum stage, which lets them assume there’s an underlying hidden structure in those items and facts, essentially modeling them as coming from a single cluster.
Jane: That part is interesting because it means they can select facts that belong to the majority class if that structure makes sense for the problem.
Tom: It helps explain their decisions because they aren't just guessing; they are selecting based on that assumed underlying structure of what’s possible.
Jane: So, what does this mean for us? It means we can use existing large models to handle complex logical deductions over many facts in a structured way, and the results are more interpretable because you can trace back through those explicit retrieval steps.
Tom: It makes the system less sensitive to how you format your prompt because the heavy lifting of probabilistic inference is happening outside of the LLM’s main generation step.
Jane: That robustness is key when we are trying to deploy these things in real applications where prompt formatting can easily get messy or accidentally confusing.
Tom: And they showed this method performing better than older chain-of-thought methods on thirteen different hard tasks from BIG-bench, often even with smaller model variants like GPT two small.
Jane: So, if you’re a developer trying to build a system that needs to reason over multiple items, this shows you that breaking the problem down into parallel retrieval and structured inference is a much more promising way forward than just one big prompt.
Tom: The main point is that we can get more reliable reasoning from models of existing scale when we use a controller that allows for this kind of probabilistic inference to happen externally.
Jane: It’s about controlling the reasoning process by separating the search and the final decision-making into two distinct, measurable steps.
Tom: And while they pointed out that the cost scales with those parallel calls and you still need human effort to set up those specific operations for a new task, it’s a solid framework for getting structured output.
Jane: It’s definitely a way to enhance the capability of current models without needing a massive retraining effort just to get better at multi-object reasoning.
Tom: So, the idea is that this two-stage paradigm—Think for retrieval and Sum for inference—is a more robust way to tackle complex set problems. But we still have those scaling costs and manual setup hurdles to deal with before it’s ready for mass deployment.
Conclusion: Tom: So, to wrap up, ThinkSum is a two-stage method that uses parallel LLM calls for association and then uses probabilistic inference for aggregation to reason over sets of objects or facts.
Jane: Basically, it shows we can use existing large language models effectively by giving them a smart controller that handles the set reasoning in a structured way instead of just letting them generate one long answer.
Tom: It’s about making the model less sensitive to prompt design because the probabilistic math happens externally, which gives us more interpretable predictions.
Jane: And they achieved improvements on thirteen hard tasks from BIG-bench, proving that this set-based thinking actually beats some of the state of the art using GPT family models.
Tom: The main implication is that we can get better logical reasoning out of existing AI by controlling the inference step outside of the model itself.
Jane: It suggests a direction for building more robust systems, where structure and explicit retrieval steps are prioritized over just hoping the LLM writes a perfect narrative.
Tom: We still have those scaling costs to worry about though, because the computation scales with how many parallel calls you make for each task.
Jane: And there's that necessary human effort involved in setting up those specific operations for every new task you want to solve.
Tom: So, it’s a promising approach for enhancing reasoning capabilities with models of existing scale, but we gotta keep an eye on those scaling costs and setup overhead when we move this into production.
Jane: It definitely points toward needing these kinds of structured controllers if we want AI to handle complex logic reliably in the real world.
Lu: From my side, I see the latent variable model idea as really interesting because it gives us a way to assume an underlying structure for those facts, which could change how we think about knowledge representation in these models.
Meng: For me, at the production level, the key is that you have to be able to predict exactly how many parallel calls you’re going to make so you can manage the compute budget and avoid running into unexpected bottlenecks.
Lalam: I'm sure this works well because my internal literature shows that when we decompose a complex reasoning task into these retrieval and aggregation steps, the confidence in the final prediction is much higher than if I just tried to do it all in one pass.
Batu Ozturkler, Nikolay Malkin, Zhen Wang, Nebojsa Jojic
Stanford University · Université de Montréal · Ohio State University · Microsoft Research
cs.CL
Submitted: 2022-10-04
Updated: 2026-11-18
Comments: ACL 2023
DOI: 10.18653/v1/2023.acl-long.68
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) have a substantial capacity for high-level analogical reasoning: reproducing patterns in linear text that occur in their training data (zero-shot evaluation) or in the…
Key concepts
- Think Stage
- The first stage where an LLM is queried in parallel over a set of phrases derived from the prompt or auxiliary calls. This step creates associations between text spans and sets of strings, such as generating examples or performing string manipulations like 'Order inversion', to build a pool of potential knowledge for later inference.
- Sum Stage
- The second stage where the results from the Think stage are aggregated using various probabilistic inference techniques like majority voting or product aggregation. This step takes the LLM outputs and combines them mathematically to produce the final, reasoned prediction.
- Probabilistic Inference
- The mathematical process used in the Sum stage to aggregate multiple LLM outputs into a single answer. Techniques like mixture aggregation or product aggregation are employed to find the most likely outcome from a set of possibilities, effectively performing reasoning outside of direct LLM generation.
- Latent Variable Models
- A technique used in the Sum stage where items and facts are modeled as being generated from an underlying, unobserved structure or 'latent class'. This allows the system to select facts that belong to the majority class, helping to explain decisions through this structured knowledge.
Terminology
Summary
Large language models (LLMs) have a substantial capacity for high-level analogical reasoning: reproducing patterns in linear text that occur in their training data (zero-shot evaluation) or in the provided context (few-shot in-context learning) However, recent studies show that even the more advanced LLMs fail in scenarios that require reasoning over multiple objects or facts and making sequences of logical deductions We propose a two-stage probabilistic inference paradigm, ThinkSum, which reasons over sets of objects or facts in a structured manner In the first stage (Think – retrieval of associations), a LLM is queried in parallel over a set of phrases extracted from the prompt or an auxiliary model call In the second stage (Sum – probabilistic inference or reasoning), the results of these queries are aggregated to make the final prediction We demonstrate the possibilities and advantages of ThinkSum on the BIG-bench suite of LLM evaluation tasks, achieving improvements over the state of the art using GPT-family models on thirteen difficult tasks, often with far smaller model variants Our results suggest that because the probabilistic inference in ThinkSum is performed outside of calls to the LLM, ThinkSum is less sensitive to prompt design, yields more interpretable predictions, and can be flexibly combined with latent variable models to extract structured knowledge from LLMs Overall, our proposed paradigm represents a promising approach for enhancing the reasoning capabilities of LLMs. p. 0 com
Abstract
Large language models (LLMs) have a substantial capacity for high-level analogical reasoning: reproducing patterns in linear text that occur in their training data (zero-shot evaluation)
or in the provided context (few-shot in-context
learning). However, recent studies show that
even the more advanced LLMs fail in scenarios that require reasoning over multiple objects
or facts and making sequences of logical deductions.Where the paper says it
ThinkSum Paradigm and Mechanism
The ThinkSum paradigm is a two-stage process where the first stage, Think,
involves querying an LLM in parallel over a set of phrases extracted from the prompt or an auxiliary model call The second stage, Sum,
aggregates the results of these queries to produce the final prediction through probabilistic inference. The Think step involves creating an association of text spans with sets of strings, which may involve generation from a language model or solely a scoring mechanism resulting in a matrix of probabilities Examples of the Think method include elementary string manipulations such as Order inversion
and Premise erasure
, as well as example generation, list extension, and fact generation.
Inference Aggregation Techniques
The Sum stage employs various techniques to aggregate the results obtained from the Think step, including several elementary inference methods These include The term inside the
sum is a case of Product aggregation: the LLM
likelihoods of all strings in the set are multiplied. In summary, our solution to this task involves
composition of two Think operations – a Translation into a set of equations and then Substitution
of numbers in place of item names – and two Sum
operations – a Product aggregation followed by
a Mixture aggregation.Majority/minority vote (argmin/argmax)
, Ratio of likelihoods
, and Mixture (average) aggregation
. Product aggregation is also utilized, such as in the LOGICAL DEDUCTION task where it involves multiplying the LLM likelihoods of all strings in a set. p. 6Where the paper says it
Performance Across Diverse Tasks
ThinkSum has been demonstrated on thirteen difficult tasks from BIG-bench, achieving improvements over the state of the art using GPT-family models on thirteen difficult tasks, often with far smaller model variants For instance, in the LOGICAL DEDUCTION task with 500 problems where N = 5 objects, ThinkSum yields an accuracy of 77%, besting the average human performance Furthermore, in the ODD ONE OUT task, near-average-human accuracy is maintained for all model sizes from GPT-2 Small to the largest GPT-3 model.
Comparison with Existing Methods
An important difference between ThinkSum and existing chain-of-thought-like prompt engineering methods is that our reasoning step is not reduced to a generation problem for the LLM, but is performed as a probabilistic inference external to the LLM This external probabilistic inference reduces vulnerability to features of the prompt, such as accidental distraction of the LLM by spurious patterns ThinkSum outperforms chain-of-thought prompting on all tasks with all variants except KNOWN UNKNOWNS with the davinci variant.
Conclusion and Future Directions
The main claim of this paper is that more is possible with LLMs of existing scale when they are used in concert with a wise controller that allows for probabilistic inference ThinkSum is more robust to prompt design, yields more interpretable predictions, and can be combined with many probabilistic inference approaches to tackle a diverse set of tasks. The computation cost for ThinkSum scales with the number of parallel calls to the LLM, which is determined for each task by the number of Think prompts used and the number of objects for which likelihood computations are required at the Sum stage. p. 9 Firstly, as the number of objects or facts that
are reasoned over increases, the computation cost
will also rise. However, increasing the number of
objects will also make the task harder, and direct
prompting may cease to work at all (as we indeed
observe in BIG-bench results, such as LOGICAL
DEDUCTION with more than five objects), while
ThinkSum offers a generalizable methodology, as
the atomic Think operations do not increase in
complexity as the number of objects grows. We have described ThinkSum as a general twostage paradigm, with an external inference step. This generality aims to facilitate the adaptation of
ThinkSum to new tasks, with minimal modifications to the Think and Sum steps.Where the paper says it
Where the paper says it
Acknowledgments and Limitations
The authors thank Alexandros Graikos, Sudha Rao, and Alessandro Sordoni for valuable discussions A limitation noted is that as the number of objects or facts that are reasoned over increases, the computation cost will also rise Additionally, when solving a new task, it is necessary to expend human effort to select specific operations in each step.
Ethics and Impact Statement
Control of LLMs’ reasoning in the way we have proposed can potentially mitigate such bias due both to the decomposition of the argumentation process into interpretable fact-retrieval steps and to the averaging effect of smoothing out spurious triggers when aggregating many hypotheses and reasoning chains.
References
Yoshua Bengio. 2017. The consciousness prior. p. 9 References
Yoshua Bengio. The consciousness prior. References
Yoshua Bengio. The consciousness prior. References
Yoshua Bengio. The consciousness prior. The consciousness prior. arXiv
preprint arXiv:1709. The consciousness prior. arXiv
preprint arXiv:1709. The consciousness prior. arXiv
preprint arXiv:1709. arXiv
preprint arXiv:1709. Tom Brown, Benjamin Mann, Nick Ryder, Melanie
Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind
Neelakantan, Pranav Shyam, Girish Sastry, Amanda
Askell, Sandhini Agarwal, Ariel Herbert-Voss,
Gretchen Krueger, Tom Henighan, Rewon Child,
Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens
Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack
Clark, Christopher Berner, Sam McCandlish, Alec
Radford, Ilya Sutskever, and Dario Amodei. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi
Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi
Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi
Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language
models for interpretable logical reasoning. Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language
models for interpretable logical reasoning. Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language
models for interpretable logical reasoning. 70
Table 2: Different alternatives of probabilistic reasoning
with ThinkSum for solving ODD ONE OUT: list of
words, latent variable model, minority voting. (EM; Dempster et al. 70
Table 2: Different alternatives of probabilistic reasoning
with ThinkSum for solving ODD ONE OUT: list of
words, latent variable model, minority voting. (EM; Dempster et al. 70
Table 2: Different alternatives of probabilistic reasoning
with ThinkSum for solving ODD ONE OUT: list of
words, latent variable model, minority voting. (EM; Dempster et al. Laird, and D. Maximum likelihood from incomplete data via the
EM algorithm. Laird, and D. Maximum likelihood from incomplete data via the
EM algorithm. Laird, and D. Maximum likelihood from incomplete data via the
EM algorithm.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Additional Tasks and Details
The paper includes case studies on several BIG-bench tasks, such as Conceptual combinations, Odd one out, Logical deduction, and Phrase relatedness Detailed experimental parameters include maximum generation length of 100 for example and list generation tasks The experiments are performed using GPT-3 variants and GPT-2 models. The paper also details a latent variable model approach to select facts that characterize the majority class for ODD ONE OUT
Inference Cost Analysis
The inference cost for ThinkSum scales with the number of parallel calls to the LLM, which is determined for each task by the number of Think prompts used and the number of objects for which likelihood computations are required at the Sum stage In some cases, ThinkSum is faster than chain-of-thought prompting due to its ability to perform parallel calls to the LLM The inference cost increase is marginal for tasks where the number of Think prompts is not typically high and the prompts are short.
Inference as Reasoning
The work generalizes such approaches to perform arbitrary probabilistic inference outside of the LLM The authors model items i ∈ I and facts f ∈ F as being generated from a latent class c ∈ 0, 1 using an expectation-maximization (EM) algorithm to derive P(i c) and P(f c) This inference procedure can be seen as a Sum (inference) operation and can be applicable in other tasks of similar structure.
Examples of Inference Procedures
The paper details specific inference procedures for different tasks, such as the translation into a set of inequalities and then substitution of numbers in place of item names This involves composition of two Think operations – a Translation into a set of equations and then Substitution of numbers in place of item names – and two Sum operations – a Product aggregation followed by a Mixture aggregation.
Results Summary
For the 500 LOGICAL DEDUCTION problems with N = 5 objects, ThinkSum yields an accuracy of 77% (see Table 1), besting the average human performance The final result for INVENTED WORDS was Borger folpts are pet toys
Table Summaries
Table 3 shows the comparison of ThinkSum with algorithms that append auxiliary knowledge as an oracle ‘reasoning chain’ Table 4 compares ThinkSum with chain-of-thought prompting approaches Table D.1 lists examples for the studied BIG-bench tasks The results show that ThinkSum outperforms auxiliary knowledge and chain of thought on LOGICAL DEDUCTION. p. 7 Above we have compared ThinkSum
with approaches that add extracted (from an auxiliary LM call), not generated (within the LM’s linear workspace) token sequences after the prompt,
for the ODD ONE OUT, PHRASE RELATEDNESS,
and LOGICAL DEDUCTION tasks (see Table 3). With suitable examples, it may be possible for
a CoT approach to replace the Think phase, by
learning from demonstrations to generate the appropriate knowledge, and parts of the Sum phase,
although inference over parallel evaluations of the
LLM is no longer possible.Where the paper says it
Figure Analysis
Figure 3 shows details for LOGICAL DEDUCTION, including a demonstration for the Think prompt, and an example LLM output Figure 4 shows how performance varies with the size of the LLM used for GPT-3, auxiliary knowledge and ThinkSum on ODD ONE OUT Figure D.1 shows the probabilities of different (in)equalities according to GPT-3 text-davinci-002 The results show that ThinkSum dramatically improves over much larger largest zero- or few-shot models with or without auxiliary knowledge. p. 5 2 (left) shows how performance varies with
the size of the LLM used for GPT-3, auxiliary
knowledge and ThinkSum on ODD ONE OUT. Even with GPT-2 Small, ThinkSum dramatically
improves over much larger largest zero- or few-shot
models with or without auxiliary knowledge.Where the paper says it
Final Conclusion
The two-step paradigm allows operating over sets instead of manipulating the prompt itself, preventing sensitivity to prompt format during the probabilistic inference in ThinkSum As a result, ThinkSum is more robust to prompt design, yields more interpretable predictions, and can be combined with many probabilistic inference approaches to tackle a diverse set of tasks. The paper concludes that more is possible with LLMs of existing scale when they are used in concert with a wise controller that allows for probabilistic inference. p. 2 As is the case with humans, there
is always the next frontier of problems where a
trained model with remarkable ‘intuition’ needs to
be slowed down. The main claim of this paper is
that more is possible with LLMs of existing scale
when they are used in concert with a wise controller
that allows for probabilistic inference.Where the paper says it
Limitations and Extensions
The authors acknowledge that as the number of objects or facts that are reasoned over increases, the computation cost will also rise A limitation is that when solving a new task, it is necessary to expend human effort to select specific operations in each step Future work could focus on automating the prompt engineering procedure for overcoming this limitation. The authors suggest that applying ThinkSum to more recent instruction-tuned models such as Flan-PaLM, text-davinci-003, ChatGPT, and GPT-4 may be beneficial The paper foresees no direct or immediate societal impacts arising from this work. p. 9 ,
2022), text-davinci-003, ChatGPT, and GPT-4,
which seem more capable of robustly performing
Think steps. Ethics and impact statement
We foresee no direct or immediate societal impacts
arising from this work.Where the paper says it
Yoshua Bengio. 2017. The consciousness prior. arXiv preprint arXiv:1709.08568 Tom Brown et al. p. 9 arXiv
preprint arXiv:1709. Tom Brown, Benjamin Mann, Nick Ryder, Melanie
Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind
Neelakantan, Pranav Shyam, Girish Sastry, Amanda
Askell, Sandhini Agarwal, Ariel Herbert-Voss,
Gretchen Krueger, Tom Henighan, Rewon Child,
Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens
Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack
Clark, Christopher Berner, Sam McCandlish, Alec
Radford, Ilya Sutskever, and Dario Amodei. arXiv
preprint arXiv:1709. Tom Brown, Benjamin Mann, Nick Ryder, Melanie
Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind
Neelakantan, Pranav Shyam, Girish Sastry, Amanda
Askell, Sandhini Agarwal, Ariel Herbert-Voss,
Gretchen Krueger, Tom Henighan, Rewon Child,
Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens
Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack
Clark, Christopher Berner, Sam McCandlish, Alec
Radford, Ilya Sutskever, and Dario Amodei. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi
Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi
Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Figure 3 shows details for LOGICAL DEDUCTION, including a demonstration for the Think prompt, and an example LLM output Figure 4 shows how performance varies with the size of the LLM used for GPT-3, auxiliary knowledge and ThinkSum on ODD ONE OUT Figure D.1 shows the probabilities of different (in)equalities according to GPT-3 text-davinci-002 The results show that ThinkSum dramatically improves over much larger largest zero- or few-shot models with or without auxiliary knowledge The results show that ThinkSum is 23% faster for PHRASE RELATEDNESS compared to chain-of-thought approaches with 5 facts generated using InstructGPT. p. 5 2 (left) shows how performance varies with
the size of the LLM used for GPT-3, auxiliary
knowledge and ThinkSum on ODD ONE OUT. Even with GPT-2 Small, ThinkSum dramatically
improves over much larger largest zero- or few-shot
models with or without auxiliary knowledge.Where the paper says it
The two-step paradigm allows operating over sets instead of manipulating the prompt itself, preventing sensitivity to prompt format during the probabilistic inference in ThinkSum As a result, ThinkSum is more robust to prompt design, yields more interpretable predictions, and can be combined with many probabilistic inference approaches to tackle a diverse set of tasks The paper concludes that more is possible with LLMs of existing scale when they are used in concert with a wise controller that allows for probabilistic inference.
The authors acknowledge that as the number of objects or facts that are reasoned over increases, the computation cost will also rise A limitation is that when solving a new task, it is necessary to expend human effort to select specific operations in each step Future work could focus on automating the prompt engineering procedure for overcoming this limitation
Yoshua Bengio. 2017. The consciousness prior. arXiv preprint arXiv:1709.08568 Tom Brown et al. 2020. Language models are few-shot learners Hyung Won Chung et al. 2023. Scaling instruction-finetuned language models Antonia Creswell et al. 2023. Selection-inference: Exploiting large language models for interpretable logical reasoning A P. Dempster et al. 1977. Maximum likelihood from incomplete data via the EM algorithm David Dohan et al. 2022.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Probabilistic reasoning over sets using ThinkSum allows models to reason over multiple objects or facts by performing a two-stage process: Think (retrieval of associations via auxiliary model calls) and Sum (probabilistic inference). This structure makes the system
less sensitive to prompt design, yields more interpretable predictions,
which enhances robustness compared to direct prompting. -
The Think stage can be implemented using elementary string manipulations such as
Order inversion
orPremise erasure,
and list generation prompts like “List facts about cats,” enabling the model to create sets of examples for later inference. This allows the system to performfast thinking / association / knowledge retrieval step: creating an association of text spans with sets of strings.
-
The Sum stage utilizes various probabilistic inference methods, including
Mixture aggregation
andProduct aggregation,
which aggregate generated strings or probabilities to produce the final answer. For tasks like Conceptual combinations, this involves forming a mixture model over substitutions and normalizing likelihoods to select the most plausible sentence. -
Latent variable models can be incorporated into the Sum stage to assume an underlying structure in items and facts, modeling them as being generated from a latent class such that
the facts and items belonging to a cluster can be freely interchanged.
This helps explain decisions by selecting facts that characterize the majority class. -
The system can perform structured inference for logical deduction tasks by translating problems into sets of mathematical inequalities and then performing
Product aggregation followed by a Mixture aggregation
over all possible mappings to determine the correct ordering.
Abstract
Large language models (LLMs) have a substantial capacity for high-level analogical reasoning: reproducing patterns in linear text that occur in their training data (zero-shot evaluation) or in the provided context (few-shot in-context learning). However, recent studies show that even the more advanced LLMs fail in scenarios that require reasoning over multiple objects or facts and making sequences of logical deductions. We propose a two-stage probabilistic inference paradigm, ThinkSum, which reasons over sets of objects or facts in a structured manner. In the first stage (Think - retrieval of associations), a LLM is queried in parallel over a set of phrases extracted from the prompt or an auxiliary model call. In the second stage (Sum - probabilistic inference or reasoning), the results of these queries are aggregated to make the final prediction. We demonstrate the possibilities and advantages of ThinkSum on the BIG-bench suite of LLM evaluation tasks, achieving improvements over the state of the art using GPT-family models on thirteen difficult tasks, often with far smaller model variants. We also compare and contrast ThinkSum with other proposed modifications to direct prompting of LLMs, such as variants of chain-of-thought prompting. Our results suggest that because the probabilistic inference in ThinkSum is performed outside of calls to the LLM, ThinkSum is less sensitive to prompt design, yields more interpretable predictions, and can be flexibly combined with latent variable models to extract structured knowledge from LLMs. Overall, our proposed paradigm represents a promising approach for enhancing the reasoning capabilities of LLMs.
Sources
- The Consciousness Prior
- PaLM: Scaling Language Modeling with Pathways
- Scaling Instruction-Finetuned Language Models
- Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning
- Language Model Cascades
- Inductive Biases for Deep Learning of Higher-Level Cognition
- Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
- Decomposed Prompting: A Modular Approach for Solving Complex Tasks
- Large Language Models are Zero-Shot Reasoners
- A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation
- Show Your Work: Scratchpads for Intermediate Computation with Language Models
- Training language models to follow instructions with human feedback
- Scaling Language Models: Methods, Analysis & Insights from Training Gopher
- Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
- Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
- Rationale-Augmented Ensembles in Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- An Explanation of In-context Learning as Implicit Bayesian Inference
- The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning
Related papers
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data
- BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation
- Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking
- Self-Modeling Interventions can Modulate Emergent Misalignment