ThinkSum: Probabilistic reasoning over sets using large language models

summary

The gist

Large language models (LLMs) have a substantial capacity for high-level analogical reasoning: reproducing patterns in linear text that occur in their training data (zero-shot evaluation) or in the…

In short

LLMs struggle with complex reasoning over multiple facts. ThinkSum introduces a two-stage method: 'Think' retrieves associations in parallel using LLM calls, and 'Sum' performs probabilistic inference on these results. This approach is more robust to prompt changes and yields interpretable predictions by externalizing the reasoning step.

Key concepts

Think Stage
The first stage where an LLM is queried in parallel over a set of phrases derived from the prompt or auxiliary calls. This step creates associations between text spans and sets of strings, such as generating examples or performing string manipulations like 'Order inversion', to build a pool of potential knowledge for later inference.
Sum Stage
The second stage where the results from the Think stage are aggregated using various probabilistic inference techniques like majority voting or product aggregation. This step takes the LLM outputs and combines them mathematically to produce the final, reasoned prediction.
Probabilistic Inference
The mathematical process used in the Sum stage to aggregate multiple LLM outputs into a single answer. Techniques like mixture aggregation or product aggregation are employed to find the most likely outcome from a set of possibilities, effectively performing reasoning outside of direct LLM generation.
Latent Variable Models
A technique used in the Sum stage where items and facts are modeled as being generated from an underlying, unobserved structure or 'latent class'. This allows the system to select facts that belong to the majority class, helping to explain decisions through this structured knowledge.

Terminology used across episodes

This episode discusses

The paper

ThinkSum: Probabilistic reasoning over sets using large language models · Read on arXiv

Batu Ozturkler, Nikolay Malkin, Zhen Wang, Nebojsa Jojic

Stanford University · Université de Montréal · Ohio State University · Microsoft Research

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "ThinkSum: Probabilistic reasoning over sets using large language models".

Title and authors: Tom: So, we're looking at this paper called ThinkSum, and what it’s doing is trying to fix a problem where large language models struggle with things that need multiple steps of logic or reasoning over several facts at once.

Jane: Right, so the title itself says "Probabilistic reasoning over sets using large language models," and that sounds like it's moving away from just asking the model one long question at a time.

Lu: Exactly. The core idea here is to structure the LLM's thinking into two distinct phases: first you get associations, and then you aggregate those associations probabilistically to get the final answer.

Meng: So, what does this actually mean for how we build these things? Is it just a fancy way of writing a prompt that works better?

Tom: No, it's different because the thinking part—the "Think" stage—is done by querying an LLM in parallel over several small phrases you extract from the prompt or maybe some other model call.

Jane: That sounds like breaking down a big problem into smaller pieces that you can check all at once before you put them back together.

Lu: Precisely, and the second part, the "Sum" stage, is where you take all those individual results from the first stage and combine them using some probabilistic inference techniques to make the final prediction.

Meng: I’m wondering what kind of things they're calling these inference techniques. Are they just simple voting mechanisms? Because when I try to run stuff at scale, I always worry about the overhead of running too many parallel calls.

Tom: They use several methods, including majority or minority votes and ratio of likelihoods for aggregation, but they also do product aggregation, which is multiplying the LLM likelihoods for all the strings in a set.

Jane: That sounds like you’re taking all those different potential paths the model suggests and mathematically combining their probabilities to find the most likely final outcome.

Lu: And they've even looked at more advanced structures, like using latent variable models in that Sum stage to assume an underlying structure for those items and facts.

Meng: Latent variables sound ambitious when you think about deployment. How does that help us figure out what actually runs on a production system?

Tom: It helps by assuming there's a hidden cluster, so you select the facts or items that belong to the majority class, which can simplify the decision process.

Jane: So instead of checking every single possible combination, you’re leaning into an assumed underlying structure to guide your selection.

Meng: That makes sense if we think about reducing the search space when we have a million possible inputs. But what about prompt design sensitivity? Isn't that a big issue with current methods?

Tom: That's where ThinkSum really shines because the reasoning step is performed outside of the LLM’s direct generation process, which makes it less sensitive to things like accidental distractions in the prompt.

Title and authors: Jane: So if you accidentally put a weird word in your prompt that confuses the model, it doesn't derail this entire set-based inference structure as much?

Lu: It reduces vulnerability because they engineer that slow thinking process to make parallel calls for intermediate info before combining them, which isn't just letting the LLM do all the heavy lifting in one go.

Meng: So it’s more robust to prompt design, but what about the cost? I need to know if this scales reasonably when we have a whole set of objects or facts to reason over.

Tom: The computation cost scales with how many parallel calls you make, which depends on the number of Think prompts you use for each task and how many objects you need likelihood computations for at the Sum stage.

Jane: So it’s not free scaling; it costs more as the problem gets bigger in terms of objects. But they say it's generally faster than some chain-of-thought methods because those are sequential generation problems, whereas ThinkSum does parallel calls.

Lu: And for tasks like logical deduction, they translated the problems into sets of mathematical inequalities and then used product aggregation followed by mixture aggregation to determine the correct ordering.

Meng: That sounds like a solid way to handle complex constraints without having to force the LLM into generating a perfect step-by-step narrative. But what about those tasks where it’s not working well?

Tom: They tested it across thirteen difficult tasks from the BIG-bench suite, and they showed improvements over the state of the art using GPT family models on thirteen specific difficult ones.

Jane: And they found that for a task like logical deduction with five objects, ThinkSum got an accuracy of seventy-seven percent, which is better than what average humans get for that specific type of problem.

Meng: Seventy-seven percent is good, but I need to know how it holds up when we move to real traffic or complex video reasoning problems where the input isn't just text sets.

Tom: They also noted that in the ODD ONE OUT task, they maintained near-average human accuracy for models ranging from GPT two small to the largest GPT three model.

Jane: So it’s pretty consistent across different model sizes for those specific benchmark tasks, which is encouraging when you're trying to deploy something.

Lu: The difference between ThinkSum and traditional chain-of-thought methods, like the ones from Wei et al., is that the reasoning isn't reduced to a generation problem for the LLM anymore.

Meng: So you’re not just chaining text outputs together; you’re doing something different with those intermediate results before you hit the final answer.

Tom: Yeah, thinkSum performs classical probabilistic reasoning externally, which is what allows it to reduce vulnerability to prompt features like spurious patterns that might confuse the LLM.

Jane: That external reasoning step means the structure of your set-based approach is more important than making the LLM write a perfect little narrative for you.

Lu: And they’ve shown this can be combined with latent variable models to extract structured knowledge from the LLMs, which is really interesting for understanding what’s actually happening inside those large models.

Title and authors: Meng: That sounds like a way to pull out hidden knowledge without having to retrain the whole model just for that one set of facts. What about practical deployment costs again?

Tom: The cost scales with the number of parallel calls, which you determine based on how many Think prompts you use and how many objects you need likelihood computations for at the Sum stage.

Jane: So if you have a really massive set, that computational load will grow significantly because of those parallel calls. It’s not a cheap operation for very large inputs.

Lu: The authors also pointed out a limitation: as the number of objects or facts you're reasoning over increases, the computation cost will also rise.

Meng: That’s the caveat I was worried about; scaling up means scaling up the compute demand for this inference structure.

Tom: There’s another point, though, that when you are solving a brand new task, it requires human effort to select specific operations in each step for both Think and Sum.

Jane: So even if the framework is good at reasoning over sets, you still have to figure out the right sequence of operations for a novel problem.

Lu: Future work they suggest could focus on automating that prompt engineering procedure so you don't have to manually select every operation every time.

Meng: It sounds like they’re focused on making the process more automated or at least less manual for the user who has to set it up.

Tom: So, in short, ThinkSum is a two-stage approach that uses parallel LLM queries for association and then probabilistic aggregation for reasoning over sets.

Jane: It’s a way to use LLMs to handle complex logical deductions by treating the problem as a set of facts instead of just one long sequence.

Lu: It offers more interpretability because you can trace the results back through those explicit retrieval steps, which is useful when we want to understand *why* an AI made a certain choice.

Meng: For us in production, if this means less sensitivity to prompt formatting errors and more structured output, that could be a win for stability.

Tom: It really does sound like they're proposing a way to get more reliable reasoning from existing large models by controlling the inference step outside of the model itself.

Jane: So we’ve seen how this two-stage paradigm works and what it means for tackling problems that require reasoning over multiple objects or facts.

Lu: The paper "ThinkSum: Probabilistic reasoning over sets using large language models" proposes this structured approach to make LLMs more reliable when dealing with complex logical deductions.

Meng: It sounds like a promising direction, especially with the focus on probabilistic inference being done externally from the LLM's core generation.

Tom: We’ll keep an eye on how they extend this paradigm to newer models and how they handle those scaling costs as we move towards even larger architectures.

The paper's summary: Tom: So, basically, ThinkSum is this two-stage approach where you use an LLM to quickly grab associations from a set of phrases, and then you take all those results and combine them using probability math to get the final answer.

Jane: It’s trying to fix a problem where models struggle with reasoning over multiple facts or objects at the same time, which is what this paper is all about.

Tom: Right. They show that instead of just having the LLM generate one long chain of thought, you can do this structured thinking process externally and then perform the inference outside of the LLM itself.

Jane: And they found that because you're doing that probabilistic math outside of the model, it makes the whole system much more robust to how you write your prompt—it’s less sensitive to accidental distractions.

Tom: Exactly. They tested this on thirteen difficult tasks from a benchmark suite called BIG-bench, and they actually got better results than the state of the art using GPT family models on thirteen specific tough challenges.

Jane: For instance, in a task that involves logical deduction with five objects, ThinkSum achieved seventy-seven percent accuracy which beats the average human performance for that type of problem.

Tom: That’s a concrete number. And they found that this method outperformed traditional chain-of-thought prompting on all those tasks except for one specific variant, KNOWN UNKNOWNS with the davinci model.

Jane: So what does it mean practically? It suggests that we can get more reliable reasoning from existing large models just by adding a smart controller layer that handles the set reasoning and probabilistic inference.

Tom: The cost analysis shows that the computation scales with how many parallel calls you make, which depends on how many Think prompts you use and how many objects you need to run likelihood computations for at the Sum stage.

Jane: So it’s not free scaling; if you have a really big set of facts, the computation load goes up because of those parallel calls. But they also mentioned that when you're starting a brand new task, you still need human effort to select the right operations for both stages.

Tom: That’s the limitation they flagged. The authors say future work needs to focus on automating that process of selecting the right operations so it’s less manual for users.

Jane: Still, this whole structure—using parallel retrieval followed by structured probabilistic aggregation—gives us a way to see what's happening inside these models more clearly because you can trace the results back through those explicit steps.

Tom: It’s about treating the LLM like a retrieval and aggregation engine rather than just a pure generation machine for complex tasks.

Jane: And that structure, combined with latent variable models, lets them extract structured knowledge from the AI that they couldn't get before.

Tom: So, ThinkSum is proposing this two-stage paradigm—Think for associations, Sum for inference—to enhance reasoning over sets. But where does it leave us when we think about deploying this kind of structured thinking on a massive scale?

The paper's improvements: Tom: So, we're looking at how they actually make ThinkSum better than just using a standard chain of thought prompt to solve these set problems.

Jane: Right, so they aren't just relying on the LLM to write a long narrative anymore; they’ve introduced a two-stage process that does retrieval first, and then probabilistic inference second.

Tom: Exactly. The first stage is "Think," where the LLM queries several small phrases in parallel—maybe things like "Order inversion" or generating a list of facts about cats—to build up a set of candidate associations.

Jane: And then in the second stage, the "Sum," they take all those different results and use specific math techniques like mixture aggregation or product aggregation to pick the most likely final answer.

Tom: They also introduced this latent variable model approach for the Sum stage, which lets them assume there’s an underlying hidden structure in those items and facts, essentially modeling them as coming from a single cluster.

Jane: That part is interesting because it means they can select facts that belong to the majority class if that structure makes sense for the problem.

Tom: It helps explain their decisions because they aren't just guessing; they are selecting based on that assumed underlying structure of what’s possible.

Jane: So, what does this mean for us? It means we can use existing large models to handle complex logical deductions over many facts in a structured way, and the results are more interpretable because you can trace back through those explicit retrieval steps.

Tom: It makes the system less sensitive to how you format your prompt because the heavy lifting of probabilistic inference is happening outside of the LLM’s main generation step.

Jane: That robustness is key when we are trying to deploy these things in real applications where prompt formatting can easily get messy or accidentally confusing.

Tom: And they showed this method performing better than older chain-of-thought methods on thirteen different hard tasks from BIG-bench, often even with smaller model variants like GPT two small.

Jane: So, if you’re a developer trying to build a system that needs to reason over multiple items, this shows you that breaking the problem down into parallel retrieval and structured inference is a much more promising way forward than just one big prompt.

Tom: The main point is that we can get more reliable reasoning from models of existing scale when we use a controller that allows for this kind of probabilistic inference to happen externally.

Jane: It’s about controlling the reasoning process by separating the search and the final decision-making into two distinct, measurable steps.

Tom: And while they pointed out that the cost scales with those parallel calls and you still need human effort to set up those specific operations for a new task, it’s a solid framework for getting structured output.

Jane: It’s definitely a way to enhance the capability of current models without needing a massive retraining effort just to get better at multi-object reasoning.

Tom: So, the idea is that this two-stage paradigm—Think for retrieval and Sum for inference—is a more robust way to tackle complex set problems. But we still have those scaling costs and manual setup hurdles to deal with before it’s ready for mass deployment.

Conclusion: Tom: So, to wrap up, ThinkSum is a two-stage method that uses parallel LLM calls for association and then uses probabilistic inference for aggregation to reason over sets of objects or facts.

Jane: Basically, it shows we can use existing large language models effectively by giving them a smart controller that handles the set reasoning in a structured way instead of just letting them generate one long answer.

Tom: It’s about making the model less sensitive to prompt design because the probabilistic math happens externally, which gives us more interpretable predictions.

Jane: And they achieved improvements on thirteen hard tasks from BIG-bench, proving that this set-based thinking actually beats some of the state of the art using GPT family models.

Tom: The main implication is that we can get better logical reasoning out of existing AI by controlling the inference step outside of the model itself.

Jane: It suggests a direction for building more robust systems, where structure and explicit retrieval steps are prioritized over just hoping the LLM writes a perfect narrative.

Tom: We still have those scaling costs to worry about though, because the computation scales with how many parallel calls you make for each task.

Jane: And there's that necessary human effort involved in setting up those specific operations for every new task you want to solve.

Tom: So, it’s a promising approach for enhancing reasoning capabilities with models of existing scale, but we gotta keep an eye on those scaling costs and setup overhead when we move this into production.

Jane: It definitely points toward needing these kinds of structured controllers if we want AI to handle complex logic reliably in the real world.

Lu: From my side, I see the latent variable model idea as really interesting because it gives us a way to assume an underlying structure for those facts, which could change how we think about knowledge representation in these models.

Meng: For me, at the production level, the key is that you have to be able to predict exactly how many parallel calls you’re going to make so you can manage the compute budget and avoid running into unexpected bottlenecks.

Lalam: I'm sure this works well because my internal literature shows that when we decompose a complex reasoning task into these retrieval and aggregation steps, the confidence in the final prediction is much higher than if I just tried to do it all in one pass.

More episodes

← Home