SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding

summary

The gist

Large language model (LLM) research in software engineering has largely focused on tasks such as code generation and bug repair…

In short

SWE-Manager is a system designed to improve software engineering by selecting and synthesizing the best code proposal for a bug fix before writing any code. It compares multiple candidate solutions based on issue context, selects the optimal one, explains why it was chosen, and then creates a high-quality 'golden' proposal that guides implementation.

Key concepts

Proposal Selection
This is the core decision-making process where a system must compare several different potential solutions (proposals) for fixing a bug. The selection process considers the specific context of the issue and aims to choose the most appropriate candidate based on learned criteria like risk or maintainability.
Golden Proposal Synthesis
This is the final step where SWE-Manager takes the best selected proposal and refines it into a perfect, implementation-ready version. Instead of just picking one option, it synthesizes a superior guide that helps the actual coding agent write clean, correct code.
Context-Conditioned Comparison
The system learns to compare proposals not in isolation but based on the specific constraints and details provided by the issue itself. The criteria used for comparison—such as risk or fix depth—emerge directly from analyzing the unique requirements of that particular software problem.
DAPO Reinforcement Learning
This is a training technique used to optimize SWE-Manager's decision-making policy. DAPO helps the model learn the best way to select proposals by rewarding it for making good choices, ensuring it learns a robust strategy for choosing solutions.

Terminology used across episodes

This episode discusses

The paper

SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding · Read on arXiv

Boyin Tan, Haoning Deng, Junyuan Zhang, Junjielong Xu, Pinjia He, Youcheng Sun

Mohamed bin Zayed University of Artificial Intelligence

Large language model (LLM) research in software engineering has largely focused on tasks such as code generation and bug repair. In practice, teams often draft multiple candidate proposals for fixing an issue and then deliberate on one golden proposal for implementation. This selection requires not only assessing the issue's scope, impact, and urgency, but also a clear understanding of each proposal's strengths and weaknesses. A good selection could make issue resolution more reliable while reducing regression and operational risk, whereas a poor choice can increase risk and even cause unpredictable failures. We first conduct a manual study of real-world issues to characterize the rationales maintainers use when selecting among competing proposals. Motivated by these findings, we introduce SWE-Manager, a joint selection and synthesis approach that selects the best proposal and synthesizes a golden proposal. SWE-Manager is an 8B model trained via reinforcement learning (RL) to compare proposals, justify its choice, and synthesize a golden proposal for implementation. We view proposal selection as a reasoning task, mirroring how technical managers review competing proposals by weighing issue context and each proposal's solution without executing code or running tests. On the SWE-Lancer Manager benchmark, SWE-Manager achieves 53.21 selection accuracy and 57.75 earn rate, earning 152,750 dollars and outperforming strong baselines including GPT-5. To further evaluate the effectiveness of SWE-Manager in real-world issue resolution, we design the P2A framework, which simulates a real-world workflow where multiple proposals are drafted, reviewed, and a golden proposal is selected for implementation...

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding".

Jane: Here's the gist of the paper. Large language model (LLM) research in software engineering has largely focused on tasks such as code generation and bug repair, but proposal selection remains underexplored The study introduces SWE-Manager, a joint selection-and-synthesis approach that compares candidate proposals under the issue context, selects the best proposal, explains the choice, and synthesizes a golden proposal for implementation Manual Study Insights The manual study of public GitHub issue discussions revealed three properties of maintainer proposal selection First, maintainer choices are guided by recurring high-level rationales, such as Risk and Safety (42.8%), Fix Depth (29.2%), and Maintainability (16.2%) Second, proposal selection is a context-conditioned comparison among candidate proposals where criteria emerge from issue constraints Third, proposal discussion helps formulate the final implementation proposal by providing cues for refinement through clarification and comparison SWE-Manager Design and Training The hypothesis tested is that proposal selection is learnable from large-scale maintainer-selection data SWE-Manager operationalizes this by comparing the issue and all candidate proposals jointly, and it synthesizes a golden proposal that turns the selected direction into implementation-facing guidance The model is instantiated as SWE-Manager-8B, trained from Qwen3-8B using supervised fine-tuning and reinforcement learning Training involves two stages: Supervised Fine-Tuning (SFT) to teach the manager task schema and comparative reasoning, followed by Reinforcement Learning (RL) with DAPO to optimize the decision policy Performance on Benchmarks On the SWE-Lancer Manager benchmark, SWE-Manager-8B achieves 53.21% selection accuracy and a 57.75% earned rate, outperforming GPT-5 (44.15% accuracy and 37.75% earned rate) In the P2A framework on the SWE-Lancer IC benchmark, P2A with SWE-Manager achieves a 55.6% issue-resolution rate, matching GPT-5 and improving over miniSWE-agent by 7.1% Downstream Usefulness Evaluation The P2A framework evaluates the synthesized golden proposal's usefulness by using it to guide an implementation agent to produce a patch The selected-proposal control experiment shows that the synthesized golden proposal is more effective than the selected proposal in guiding the Implementation Agent, resolving 18 more issues and increasing earned reward from 58,125 to 69,875 Ablation Study Results The ablation study indicates that proposal selection improves when training requires golden-proposal synthesis rather than only best-proposal ID prediction Requiring the model to synthesize a golden proposal encourages a more explicit evaluation of candidate strengths and weaknesses, which translates into higher selection accuracy and reward Furthermore, supervised fine-tuning provides an effective warm start by teaching the model the desired reasoning structure before applying reinforcement learning via DAPO Robustness and Limitations SWE-Manager-8B remains competitive with GPT-5 on smaller proposal pools but loses accuracy as the pool grows beyond three proposals due to distribution shift A limitation is that the current SWE-Manager operates without code access, suggesting a future direction for a tool-augmented manager Future work should rebalance training toward larger proposal pools and use curriculum strategies to handle longer contexts more robustly Related Work Context Proposal selection is framed as a decision problem where the core challenge is comparing multiple candidate solutions while jointly considering functional requirements and risks This builds on prior work that treats selection as a computational ranking problem, but SWE-Manager focuses on learning issue-specific selection and synthesis over proposal pools before implementation begins The framework integrates concepts from automated program repair by studying the managerial step of selecting among competing proposals before code modification

Paper summary: Tom: So we’ve looked at SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding, which is this joint approach comparing candidates under the issue context to select the best one and then synthesizing a golden proposal for implementation <ref:2601.22956#pg1>.

Jane: The paper suggests that this selection process contains learnable patterns based on maintainer choices like risk and safety, but they also showed that synthesizing a proposal before coding is useful for guiding the actual patch generation agent

SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding: .

Lu: What this means is that instead of just treating proposal selection as a simple ranking problem, we’re learning how to handle the complex comparison between multiple technically plausible solutions based on real engineering constraints <ref:2601.22956#pg1>.

Meng: For us in production, it means we could build systems where the AI doesn't just suggest one fix but actually provides a vetted, synthesized plan for implementation that has already passed some internal quality checks

SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding: .

Lalam: I think the authors are arguing that this capability—the selection and synthesis step—is the next big thing after basic code generation or bug repair <ref:2601.22956#pg1>.

Tom: They’re proposing a framework where the model acts like a technical manager, reviewing candidates, rejecting bad ones, and then creating that final golden proposal

SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding: .

Jane: It moves us closer to an AI that understands not just *what* code to write, but *why* it’s the right code for the specific context of a bug report

SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding: .

Lu: The authors are testing this on benchmarks like SWE-Lancer and showing that the performance gains come from requiring the model to synthesize rather than just identifying the best option alone <ref:2601.22956#pg3>.

Meng: It’s a shift from a simple suggestion engine to a decision-making system that understands workflow constraints, which is what you need for real software engineering tasks

SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding: .

Lalam: The core of this work is learning issue-specific selection and synthesis over proposal pools before any actual code modification takes place

SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding: .

Conclusion: Tom: So, we're wrapping up our look at SWE-Manager and its main conclusion about proposal selection and synthesis before implementation.

Jane: It basically boils down to this: this paper tests the idea that you can learn how to pick the best code fix from a bunch of options by training an AI on real human choices made by maintainers.

Lu: Right, they're not just looking for the single best proposal; they're building a whole system that selects it and then synthesizes a perfect plan for coding it.

Meng: So, what does this mean when you look at the authors and their results? Are we looking at some small tweak or something bigger?

Tom: The authors trained an eight billion parameter model on data from maintainer selections, using supervised fine-tuning and reinforcement learning to get that synthesis skill.

Jane: And they showed it actually beats a baseline model, GPT-five on the SWE-Lancer benchmark for selection accuracy, which is about fifty-three point two one percent.

Lu: The paper highlights that requiring the model to synthesize a golden proposal instead of just picking the best one actually makes the selection process more accurate.

Meng: I'm wondering how much effort that synthesis part adds to the overall system when you're trying to deploy it at scale? Does it slow things down too much?

Tom: Well, they found that this synthesized proposal is way better for a real implementation agent; it resolves eighteen more issues and boosts the reward score from fifty-eight thousand one hundred twenty-five up to sixty-nine thousand eight hundred seventy-five.

Jane: It seems like the authors are arguing that this selection and synthesis step is a really important part of the workflow, not just an extra feature.

Lu: They're framing it as learning issue-specific selection over proposal pools before any actual code modification starts, which is different from just treating it as a ranking problem.

Meng: So, if we take that idea—learning the context for selection—what’s the next hurdle when you try to put this into a real production environment?

Tom: The paper does flag one limitation: the model's performance starts dropping once you give it more than three proposals to choose from because of distribution shift.

Jane: That means it’s strong for simpler cases, but handling really messy, complex issues with a huge number of potential fixes is still a challenge for this current setup.

More episodes

← Home