TATG: Tracking-Aware Testing Objective for LLM-based Test Generation

arXiv:2607.03194 · cs.SE · Submitted 2026-07-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "TATG: Tracking-Aware Testing Objective for LLM-based Test Generation".

Jane: Here's the gist of the paper. Complex Java methods remain challenging for automated unit test generation because achieving strong coverage and fault detection often requires satisfying branch-specific testing requirements that are not directly visible from a focal method signature Recent LLM-based frameworks, such as KTester, PANTA, and MUTGEN, leverage project context, static analysis, coverage feedback, or mutation guidance to improve test generation <ref:2607.03194#pg6> However, these approaches do not explicitly represent and track individual testing requirements across iterations <ref:2607.03194#pg6> As a result, generation and repair efforts may repeatedly target already-satisfied requirements while overlooking unresolved branches and weak oracles <ref:2607.03194#pg6> TATG introduces a unified objective representation that captures testing requirements derived from both static program analysis and dynamic feedback <ref:2607.03194#pg6> Pruning redundant reasoning tokens TATG presents a tracking-aware LLM-based unit test generation framework that unifies static program analysis and dynamic evaluation feedback through a unified objective representation This representation captures testing requirements derived from both static program analysis and dynamic feedback Each objective records its evidence, setup, action, oracle, acceptance criterion, status, and de-duplication key <ref:2607.03194#pg6> Building on this representation, TATG employs a two-stage workflow: structural rounds first improve coverage <ref:2607.03194#pg6> while hardening rounds subsequently strengthen assertions using mutation feedback to improve fault-detection capability.

Tom: How the training signal is built TATG's objective state G is a set of objectives where each objective G is represented as a tuple: G = ⟨id, src, cat, E, S, A, Q, α, status, key⟩ <ref:2607.03194#pg6> The field src records where the objective comes from in categories such as static or mutation <ref:2607.03194#pg6> Feedback-driven objectives are generated incrementally throughout iterative generation through compilation/execution diagnostics, coverage reports, and mutation testing feedback The objective state is updated after each round to record which objectives have been satisfied or remain open.

Jane: Where the gains come from TATG's two-stage generation strategy coordinates structural coverage improvement and mutation-based oracle strengthening within a unified generation process The structural stage focuses on producing executable tests and improving reachability using static objectives, while the hardening stage uses mutation feedback to strengthen assertions for code that is already reachable.

Tom: The structural stage prioritizes executable tests and hard-to-reach paths, and the hardening stage prioritizes strengthening oracles for code that is already reachable.

Paper summary: Tom: Looking back at "TATG: Tracking-Aware Testing Objective for LLM-based Test Generation" by Guancheng Wang, Qinghua Xu, and Lionel C. Briand, the core argument is that explicit tracking of testing requirements makes iterative guidance for AI test generation much more effective.

Jane: They developed a unified objective representation that captures both static analysis findings and dynamic execution feedback, which is then used in a two-stage workflow to improve coverage first then fault detection.

Lu: The implication is that instead of relying on vague prompts or coarse project knowledge, you can provide the AI with precise objectives like "exercise the branch where currToken equals Token string," which significantly improves how much code gets covered and how good those tests are at catching errors.

Meng: It sounds like this moves us away from just generating a single test suite and toward building a systematic process that refines the tests round by round based on what failed or succeeded in the last step.

Lalam: The paper shows that TATG consistently achieves higher structural coverage and mutation score than existing baselines, and for fifty methods, it matched industrial tool performance while still outperforming them in line coverage and mutation score.

Tom: It really boils down to using feedback to guide generation in a way that addresses the specific needs of the code path you're currently working on, rather than just hoping the AI guesses what’s important next.

Jane: So, TATG provides a framework where you can systematically track whether each specific requirement—like an exception translation path—has been met or left open across all iterations of test generation.

Conclusion: Tom: So we're wrapping up this look at TATG, which is about this new framework for making AI test generation better by tracking its steps instead of just guessing.

Jane: Right, so you mentioned that the authors are Wang, Xu, and Briand. What’s the main thing to get from that title?

Tom: It means they’re tracking objectives across iterations in this LLM testing setup. They built a way to record where every testing requirement comes from—whether it's static code analysis or dynamic execution feedback.

Lu: Exactly. This unified objective representation, it keeps track of the evidence, the setup, the action taken, the oracle they used for checking success, and even a status marker for each requirement.

Meng: I’m thinking about deployment here. If you're running this on real traffic at scale with that level of tracking in place—how does that affect runtime? Is it adding too much overhead?

Jane: That’s a big question, Meng. But the paper shows they use a two-stage process: first, structural rounds to get basic code coverage up using static data, and then hardening rounds using mutation feedback to fix those weak spots.

Tom: And that’s where the results really show up. They consistently beat baselines like KTester and PANTA by improving line coverage, branch coverage, and mutation score by about twenty points on average.

Lu: It’s a strong performance lift, especially because the structural stage is shown to be critical; without it, you only get about twenty-seven percent line coverage.

Jane: So what does this mean for someone who just wants to build better tests? It suggests that moving from vague hints to explicit tracking of needs makes the whole process much more reliable.

Tom: It means the AI isn't just randomly generating code; it's following a documented plan for hitting every single required test condition.

Meng: I still worry about the fidelity of those dynamic feedback loops—the compilation and execution diagnostics—but if that tracking is accurate, it’s a solid way to ensure you don't miss those edge cases that only show up during runtime.

Lu: The authors explicitly state their evaluation was fair, using the same model version and configuration across all tests to reduce the risk from just one specific setup.

Jane: So it seems TATG is pushing us toward a more structured way of teaching AI how to test complex software, moving beyond simple prompting to something much more goal-oriented.

Guancheng Wang, Qinghua Xu, Lionel C. Briand

Lero Centre for Software, University of Limerick · School of Electrical Engineering and Computer Science, University of Ottawa

cs.SE

Submitted: 2026-07-03

Updated: 2026-11-06

Code: https://github.com/paohaijiao/jquick-curl

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Complex Java methods remain challenging for automated unit test generation because achieving strong coverage and fault detection often requires satisfying branch-specific testing requirements that…

Key concepts

Unified Objective Representation
This is the core idea of TATG. It's a structured way to record every testing need derived from both static code analysis (like identifying possible branches) and dynamic feedback (like compilation errors or coverage reports). Each objective tracks its source, setup, action, oracle, and status across generations.
Structural Rounds
The first stage of TATG focuses on improving test reachability and overall code coverage. It uses static objectives to prioritize generating executable tests and exploring hard-to-reach paths in the code. This ensures the generated tests cover a broad range of execution paths before focusing on assertion strength.
Hardening Rounds
The second stage focuses on strengthening assertions using feedback from mutation testing. After structural coverage is established, this stage targets already reachable code to improve the quality of tests by making them more robust against small changes, thereby enhancing fault detection capabilities.

Terminology

Summary

Complex Java methods remain challenging for automated unit test generation because achieving strong coverage and fault detection often requires satisfying branch-specific testing requirements that are not directly visible from a focal method signature Recent LLM-based frameworks, such as KTester, PANTA, and MUTGEN, leverage project context, static analysis, coverage feedback, or mutation guidance to improve test generation <ref:2607. p. 0

Where the paper says it

briand@lero. p. 0

Where the paper says it

TATG: Tracking-Aware Testing Objective for LLM-based Test Generation Guancheng Wang∗ , Qinghua Xu∗ , Lionel C. Briand∗† ∗ Research Ireland Lero Centre for Software, University of Limerick, Limerick, Ireland † School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada guancheng.

ie Abstract—Complex Java methods remain challenging for automated unit test generation because achieving strong coverage and fault detection often requires satisfying branch-specific testing requirements that are not directly visible from a focal method signature.

03194#pg6> However, these approaches do not explicitly represent and track individual testing requirements across iterations <ref:2607. p. 0
Where the paper says it

Recent LLM-based frameworks, such as KTester, PANTA, and MUTGEN, leverage project context, static analysis, coverage feedback, or mutation guidance to improve test generation. However, these approaches do not explicitly represent and track individual testing requirements across iterations.

03194#pg6> As a result, generation and repair efforts may repeatedly target already-satisfied requirements while overlooking unresolved branches and weak oracles <ref:2607. p. 0
Where the paper says it

However, these approaches do not explicitly represent and track individual testing requirements across iterations. As a result, generation and repair efforts may repeatedly target already-satisfied requirements while overlooking unresolved branches and weak oracles.

03194#pg6> TATG introduces a unified objective representation that captures testing requirements derived from both static program analysis and dynamic feedback <ref:2607. p. 0
Where the paper says it

We present TATG, a tracking-aware LLM-based unit test generation. TATG introduces a unified objective representation that captures testing requirements derived from both static program analysis and dynamic feedback.

03194#pg6>

Pruning redundant reasoning tokens

TATG presents a tracking-aware LLM-based unit test generation framework that unifies static program analysis and dynamic evaluation feedback through a unified objective representation This representation captures testing requirements derived from both static program analysis and dynamic feedback Each objective records its evidence, setup, action, oracle, acceptance criterion, status, and de-duplication key <ref:2607.03194#pg6> Building on this representation, TATG employs a two-stage workflow: structural rounds first improve coverage <ref:2607. p. 0

Where the paper says it

Each objective records its evidence, setup, action, oracle, acceptance criterion, status, and de-duplication key, enabling fine-grained tracking of which requirements have been satisfied and which remain unresolved throughout generation. Building on this representation, TATG employs a two-stage workflow: structural rounds first improve coverage, while hardening rounds subsequently strengthen assertions using mutation feedback to improve fault-detection capability.

03194#pg6> while hardening rounds subsequently strengthen assertions using mutation feedback to improve fault-detection capability. p. 0
Where the paper says it

Each objective records its evidence, setup, action, oracle, acceptance criterion, status, and de-duplication key, enabling fine-grained tracking of which requirements have been satisfied and which remain unresolved throughout generation. Building on this representation, TATG employs a two-stage workflow: structural rounds first improve coverage, while hardening rounds subsequently strengthen assertions using mutation feedback to improve fault-detection capability.

How the training signal is built

TATG's objective state G is a set of objectives where each objective G is represented as a tuple: G = ⟨id, src, cat, E, S, A, Q, α, status, key⟩ <ref:2607.03194#pg6> The field src records where the objective comes from in categories such as static or mutation <ref:2607. p. 2

Where the paper says it

For example, an objective created from a missed branch may use the source line and branch index, while a mutation objective may use the mutant identifier reported by mutation testing. The field src records where the objective comes from: src ∈ {static, coverage, validation, oracle, mutation}.

03194#pg6> Feedback-driven objectives are generated incrementally throughout iterative generation through compilation/execution diagnostics, coverage reports, and mutation testing feedback The objective state is updated after each round to record which objectives have been satisfied or remain open. p. 4
Where the paper says it

Feedback-driven objectives are generated incrementally throughout iterative generation. Compilation and execution diagnostics, along with coverage reports, generate ValidationRepair objectives, weak assertions identified in generated tests generate OracleStrengthening objectives, and mutation testing generates MutationReachability and MutationAssertion objectives for uncovered and live mutants, respectively.

Where the gains come from

TATG's two-stage generation strategy coordinates structural coverage improvement and mutation-based oracle strengthening within a unified generation process The structural stage focuses on producing executable tests and improving reachability using static objectives, while the hardening stage uses mutation feedback to strengthen assertions for code that is already reachable. The structural stage prioritizes executable tests and hard-to-reach paths, and the hardening stage prioritizes strengthening oracles for code that is already reachable. p. 2

Where the paper says it

The structural stage prioritizes executable tests and hard-to-reach paths. The hardening stage prioritizes strengthening oracles for code that is already reachable.

Performance comparison

Results show that TATG consistently outperforms state-of-the-art baselines, including KTester and PANTA, improving line coverage, branch coverage, and mutation score by an average of 22.15, 20.14, and 37.66 percentage points <ref:2607. p. 0

Where the paper says it

14, and 37. 66 percentage points, respectively.

03194#pg6> On a randomly selected subset of 50 focal methods, TATG also achieves performance comparable to that of a proprietary industrial test generation tool while achieving higher line coverage and a higher mutation score. p. 0
Where the paper says it

66 percentage points, respectively. On a randomly selected subset of 50 focal methods, TATG also achieves performance comparable to that of a proprietary industrial test generation tool, while achieving higher line coverage and a higher mutation score.

TATG consistently achieves the best overall performance in terms of line coverage, branch coverage, and mutation score over the strongest baseline PANTA on both models <ref:2607. p. 7
Where the paper says it

The complete results are reported in Table III. Across both models, TATG consistently achieves the best overall performance in terms of line coverage, branch coverage, and mutation score while generating fewer test cases than the baselines.

03194#pg6>

Ablation study results

The ablation study demonstrates that each stage of TATG contributes to its overall performance by showing that the structural stage plays a critical role in establishing stronger structural coverage, thereby improving the reachability of subsequent mutation-based hardening. Without the structural stage, TATG achieves only 26. p. 8

Where the paper says it

Since PITest requires at least one executable test to produce mutation feedback, the first iteration is performed without mutation feedback to bootstrap test generation. Without the structural stage, TATG achieves only 26.

90% line coverage and 20. p. 0
Where the paper says it

We evaluate TATG on 141 complex Java methods, including the 110 KTester subjects and 31 additional challenging methods. Results show that TATG consistently outperforms state-of-theart baselines, including KTester and PANTA, improving line coverage, branch coverage, and mutation score by an average of 22.

45% branch coverage on Qwen. p. 8
Where the paper says it

48% and branch coverage by 4. 97% on Qwen, and by 6.

The hardening stage makes a substantial contribution by consistently improving the mutation score over the structural stage alone. p. 8
Where the paper says it

The unified objective representation enables more effective iterative guidance by tracking unresolved testing objectives and providing finegrained feedback. RQ2: Ablation Study Table III shows that the hardening stage makes a substantial contribution to the overall performance by consistently improving the mutation score over the structural stage alone.

Threats to validity

To reduce the threat of stochastic LLM outputs, all LLM-based approaches are evaluated with the same model version, configuration, prompting budget, and execution environment <ref:2607.03194#pg6> TATG explicitly targets structural coverage and mutation score when evaluating it using method-level line coverage, branch coverage, and mutation score <ref:2607. p. 9

Where the paper says it

Construct validity may be affected by whether the selected metrics accurately reflect the objectives of the proposed approach. Since TATG explicitly targets structural coverage and mutation score, we evaluate it using method-level line coverage, branch coverage, and mutation score, following prior work.

03194#pg6> External validity is limited by the benchmark and implementation scope, but TATG's objective representation is language-independent.

Conclusion

These results demonstrate that unified objective tracking enables more effective iterative guidance for LLM-based unit test generation. TATG consistently achieves higher structural coverage and mutation score than existing baselines On a randomly selected subset of 50 methods, TATG also achieves performance comparable to that of an industrial proprietary test generation tool while achieving higher line coverage and a higher mutation score. p. 9

Where the paper says it

Experimental results show that TATG consistently achieves higher structural coverage and mutation score than existing baselines. On a randomly selected subset of 50 methods, TATG also achieves performance comparable to that of an industrial proprietary test generation tool while achieving higher line coverage and a higher mutation score.

p. 9
Where the paper says it

Experimental results show that TATG consistently achieves higher structural coverage and mutation score than existing baselines. On a randomly selected subset of 50 methods, TATG also achieves performance comparable to that of an industrial proprietary test generation tool while achieving higher line coverage and a higher mutation score.

The results demonstrate that unified objective tracking enables more effective iterative guidance for LLM-based unit test generation. p. 9
Where the paper says it

CONCLUSION This paper presents TATG, an LLM-based unit test generation framework based on a unified objective representation. By representing static analysis results, execution feedback, coverage gaps, and mutation feedback as tracked objectives, TATG enables iterative guidance across generation rounds, allowing the LLM to focus on unresolved testing objectives.

p. 9
Where the paper says it

CONCLUSION This paper presents TATG, an LLM-based unit test generation framework based on a unified objective representation. By representing static analysis results, execution feedback, coverage gaps, and mutation feedback as tracked objectives, TATG enables iterative guidance across generation rounds, allowing the LLM to focus on unresolved testing objectives.

TATG consistently achieves higher structural coverage and mutation score than existing baselines On a randomly selected subset of 50 methods, TATG also achieves performance comparable to that of an industrial proprietary test generation tool while achieving higher line coverage and a higher mutation score. The results demonstrate that unified objective tracking enables more effective iterative guidance for LLM-based unit test generation.

References

[1] Z. Wang, K. Liu, G. Li, and Z. Jin, “Hits: High-coverage llm-based unit test generation via method slicing,” in Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, 2024

[2] A. Li, M. Liu, Z. Chen, Z. Pei, Z. Li, D. Dai, Y. Wang, and Z. p. 0

Where the paper says it

TATG: Tracking-Aware Testing Objective for LLM-based Test Generation Guancheng Wang∗ , Qinghua Xu∗ , Lionel C. Briand∗† ∗ Research Ireland Lero Centre for Software, University of Limerick, Limerick, Ireland † School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada guancheng.

Zheng, “Ktester: Leveraging domain and testing knowledge for more effective llm-based test generation,” arXiv preprint arXiv:2511. p. 2
Where the paper says it

LLM-based tools can reason about the source and often generate readable tests, yet a generic branch- or exceptionoriented prompt does not translate P1–P6 into distinct testing goals with required setup and measurable acceptance criteria. KTester, a recent state-of-the-art approach, combines project knowledge for test generation, however, the context remains coarse-grained (e.

14224

[3] S. Gu, N. Nashid, and A. Mesbah, “Llm test generation via iterative hybrid program analysis,” arXiv preprint arXiv:2503. p. 9

Where the paper says it

By representing static analysis results, execution feedback, coverage gaps, and mutation feedback as tracked objectives, TATG enables iterative guidance across generation rounds, allowing the LLM to focus on unresolved testing objectives. We evaluated TATG on 141 complex Java methods against representative traditional, hybrid, and LLM-based test generation approaches.

13580

[4] S. Gu, Q. Zhang, K. Li, C. Fang, F. Tian, L. Zhu, J. Zhou, and Z. Chen, “Testart: Improving llm-based unit testing via co-evolution of automated generation and repair iteration,” arXiv preprint arXiv:2408. p. 0

Where the paper says it

Building on this objective representation, TATG employs a two-stage generation strategy. The structural stage focuses on producing executable tests, improving reachability, and increasing line/branch coverage using static objectives, along arXiv:2607.

03095

[5] J. Altmayer Pizzorno and E. D. Berger, “Coverup: Effective high coverage test generation for python,” Proceedings of the ACM on Software Engineering, vol. p. 0

Where the paper says it

TATG: Tracking-Aware Testing Objective for LLM-based Test Generation Guancheng Wang∗ , Qinghua Xu∗ , Lionel C. Briand∗† ∗ Research Ireland Lero Centre for Software, University of Limerick, Limerick, Ireland † School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada guancheng.

2, no. FSE, pp. 2897–2919

[6] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pretrain, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM computing surveys

[7] A. M. Dakhel, A. Nikanjam, V. Majdinasab, F. Khomh, and M. C. Desmarais, “Effective test generation using pre-trained large language models and mutation testing,” Information and Software Technology

[8] M. Harman, J. Ritchey, I. Harper, S. Sengupta, K. Mao, A. Gulati, C. Foster, and H. Robert, “Mutation-guided llm-based test generation at meta,” in Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering

[9] G. Wang, Q. Xu, L. C. Briand, and K. p. 0

Where the paper says it

TATG: Tracking-Aware Testing Objective for LLM-based Test Generation Guancheng Wang∗ , Qinghua Xu∗ , Lionel C. Briand∗† ∗ Research Ireland Lero Centre for Software, University of Limerick, Limerick, Ireland † School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada guancheng.

Liu, “Mutation-guided unit test generation with a large language model,” arXiv preprint arXiv:2506. p. 0
Where the paper says it

On a randomly selected subset of 50 focal methods, TATG also achieves performance comparable to that of a proprietary industrial test generation tool, while achieving higher line coverage and a higher mutation score. Index Terms—Unit Test Generation, Program Analysis, Large Language Models I.

02954

[10] Q. Xu, G. Wang, L. Briand, Z. Guo, and K. Liu, “Llm-based mockless unit test generation for java,” arXiv preprint arXiv:2605. p. 0

Where the paper says it

Building on this objective representation, TATG employs a two-stage generation strategy. The structural stage focuses on producing executable tests, improving reachability, and increasing line/branch coverage using static objectives, along arXiv:2607.

26851

[11] paohaijiao, “jquick-curl: A java wrapper for libcurl,” https://github.com/paohaijiao/jquick-curl, accessed 2024-06-23

[12] Google, “Agent development kit (adk) for java,” https://github.com/google/adk-java, accessed 2024-06-23

[13] agentscope-ai, “Build production-ready ai agents in java,” https://github.com/agentscope-ai/agentscope-java, accessed 2024-06-23

[14] A2A Project, “A2a: An open-source framework for building autonomous agents in java,” https://github.com/a2aproject/a2a-java, accessed 2024-06-23

[15] “Lizard: A code complexity analyzer,” https://github.com/terryyin/lizard, accessed 2026-06-23

[16] M. Stavtsev and Y. Bugayenko, “Evaluating the dependency between cyclomatic complexity and response for class,” arXiv preprint arXiv:2410. p. 6

Where the paper says it

Consequently, the additional subjects cover a broader range of access-control constraints and dependency patterns commonly encountered in real-world Java projects. Table II summarizes the benchmark subjects, including the number of methods, cyclomatic complexity (CC), and noncomment lines of code (NLOC).

06416

[17] C. Pacheco and M. D. Ernst, “Randoop: feedback-directed random testing for java,” in Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion, 2007

[18] G. Fraser and A. Arcuri, “Evosuite: automatic test suite generation for object-oriented software,” in Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, 2011

[19] Y. Lin, Y. S. Ong, J. Sun, G. Fraser, and J. S. Dong, “Graph-based seed object synthesis for search-based unit testing,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

[20] “Jacoco,” Accessed: 2026

[21] “Pitest,” Accessed: 2026

[22] soot-oss, “Sootup: A redesign of the soot static analysis framework,” https://github.com/soot-oss/SootUp, 2023

[23] “vllm: A high-performance llm inference and serving framework,” https://vllm.ai, accessed 2024-06-23

[24] “Fireworks ai,” https://fireworks.ai/models/fireworks/deepseek-v4-flash, accessed 2024-06-23

--- Page 1 ---

TATG: Tracking-Aware Testing Objective for LLM-based Test Generation

Guancheng Wang∗, Qinghua Xu∗, Lionel C. Briand∗† Research Ireland Lero Centre for Software, University of Limerick, Limerick, Ireland † School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada guancheng. p. 0

Where the paper says it

TATG: Tracking-Aware Testing Objective for LLM-based Test Generation Guancheng Wang∗ , Qinghua Xu∗ , Lionel C. Briand∗† ∗ Research Ireland Lero Centre for Software, University of Limerick, Limerick, Ireland † School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada guancheng.

wang@ul.ie, etqinghua@gmail. p. 0
Where the paper says it

Briand∗† ∗ Research Ireland Lero Centre for Software, University of Limerick, Limerick, Ireland † School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada guancheng. qinghua@gmail.

com, lionel. p. 0
Where the paper says it

qinghua@gmail. com, lionel.

briand@lero.ie <ref:2607. p. 0
Where the paper says it

Building on this objective representation, TATG employs a two-stage generation strategy. The structural stage focuses on producing executable tests, improving reachability, and increasing line/branch coverage using static objectives, along arXiv:2607.

03194#pg6>

--- Page 2 ---

I I INTRODUCTION

Automated unit test generation has relied on random testing, search-based testing, symbolic execution, and, more recently, large language models (LLMs) <ref:2607.03194#pg6> Despite this progress, generating high-quality tests for complex Java methods remains challenging <ref:2607. p. 0

Where the paper says it

INTRODUCTION Automated unit test generation has relied on random testing, search-based testing, symbolic execution, and, more recently, large language models (LLMs). Despite this progress, generating high-quality tests for complex Java methods remains challenging.

03194#pg6> The difficulty often lies not in the size of the focal method itself, but in satisfying branch-specific testing requirements that are only partially visible from the method’s source code <ref:2607. p. 0
Where the paper says it

Despite this progress, generating high-quality tests for complex Java methods remains challenging. The difficulty often lies not in the size of the focal method itself, but in satisfying branch-specific testing requirements that are only partially visible from the method’s source code.

03194#pg6> Reaching a target behavior may require establishing a prerequisite program state, exercising a particular switch branch, triggering an exception-translation path, invoking the method through a valid entry route, or constructing assertions strong enough to detect semantic regressions <ref:2607. p. 0
Where the paper says it

The difficulty often lies not in the size of the focal method itself, but in satisfying branch-specific testing requirements that are only partially visible from the method’s source code. Reaching a target behavior may require establishing a prerequisite program state, exercising a particular switch branch, triggering an exception-translation path, invoking the method through a valid entry route, or constructing assertions strong enough to detect semantic regressions.

03194#pg6> Recent LLM-based test generation approaches have moved beyond focal-method-only prompting by incorporating project context, static analysis, coverage feedback, and mutation guidance [1], [2], [3], [4], [5], [6], [7], [8], and [9] <ref:2607. p. 0
Where the paper says it

Some of these requirements can be inferred statically, while others only become visible after generated tests are compiled, executed, and evaluated. Recent LLM-based test generation approaches have moved beyond focal-method-only prompting by incorporating project context, static analysis, coverage feedback, and mutation guidance [1], [2], [3], [4], [5], [6], [7], [8], [9].

03194#pg6> However, these signals are typically delivered to the LLM as prompts, hints, path descriptions, or post-execution feedback <ref:2607. p. 0
Where the paper says it

For example, KTester injects project structure and usage knowledge [2], PANTA uses coverage feedback to target undertested paths [3], and MUTGEN leverages mutation feedback to strengthen generated tests [9]. However, these signals are typically delivered to the LLM as prompts, hints, path descriptions, or post-execution feedback.

03194#pg6> Existing approaches do not maintain an explicit representation of individual testing requirements and their status across iterations <ref:2607. p. 0
Where the paper says it

However, these signals are typically delivered to the LLM as prompts, hints, path descriptions, or post-execution feedback. Existing approaches do not maintain an explicit representation of individual testing requirements and their status across iterations.

03194#pg6> Consequently, later rounds may revisit already-satisfied behaviors while unresolved branches, or weak oracles, remain unaddressed <ref:2607. p. 0
Where the paper says it

Existing approaches do not maintain an explicit representation of individual testing requirements and their status across iterations. Consequently, later rounds may revisit already-satisfied behaviors while unresolved branches, or weak oracles, remain unaddressed.

03194#pg6>

--- Page 3 ---

II. MOTIVATING EXAMPLE

We use a single running example to illustrate the testing problem that TATG addresses <ref:2607.03194#pg6> A useful test suite for Listing 1 must satisfy requirements that are not explicit <ref:2607. p. 1

Where the paper says it

Running example: stateful entry, finite dispatch, and exception translation. A useful test suite for Listing 1 must satisfy requirements that are not explicit.

03194#pg6> First, most paths are statedependent <ref:2607. p. 1
Where the paper says it

A useful test suite for Listing 1 must satisfy requirements that are not explicit. First, most paths are statedependent.

03194#pg6> Second, the number of nested branches is finite: P1, P2, and P3 correspond to distinct token constants, and each needs a distinct input or mock behaviour <ref:2607. p. 1
Where the paper says it

A test that simply constructs a fresh reader and calls the target may reach only the fall-through exception. Second, the number of nested branches is finite: P1, P2, and P3 correspond to distinct token constants, and each needs a distinct input or mock behaviour.

03194#pg6> Third, P5 requires exceptional predecessor behaviour: the collaborator must throw StreamException at the right call site so that the method executes the catch-rethrow block <ref:2607. p. 1
Where the paper says it

Second, the number of nested branches is finite: P1, P2, and P3 correspond to distinct token constants, and each needs a distinct input or mock behaviour. Third, P5 requires exceptional predecessor behaviour: the collaborator must throw StreamException at the right call site so that the method executes the catch-rethrow block.

03194#pg6> Finally, if the method is package-private or is normally accessed through a facade, the test must also select an executable entry route <ref:2607. p. 1
Where the paper says it

Third, P5 requires exceptional predecessor behaviour: the collaborator must throw StreamException at the right call site so that the method executes the catch-rethrow block. Finally, if the method is package-private or is normally accessed through a facade, the test must also select an executable entry route.

03194#pg6> These requirements also explain why coverage should precede mutation refinement <ref:2607. p. 1
Where the paper says it

Finally, if the method is package-private or is normally accessed through a facade, the test must also select an executable entry route. These requirements also explain why coverage should precede mutation refinement.

03194#pg6>

--- Page 4 ---

TABLE I

PROJECTION FROM SOURCE-GROUNDED EVIDENCE TO OBJECTIVES.

Category cat Evidence E from static analysis or dynamic feedback Required setup S Target action A Oracle shape Q <ref:2607. p. 5

Where the paper says it

TABLE I PROJECTION FROM SOURCE-GROUNDED EVIDENCE TO OBJECTIVES. Category cat Evidence E from static analysis or dynamic feedback Required setup S Target action A Oracle shape Q StateSetup Assignment, update, or predicate over a receiver field, e.

p. 5
Where the paper says it

TABLE I PROJECTION FROM SOURCE-GROUNDED EVIDENCE TO OBJECTIVES. Category cat Evidence E from static analysis or dynamic feedback Required setup S Target action A Oracle shape Q StateSetup Assignment, update, or predicate over a receiver field, e.

03194#pg6> Structural objectives are initialized before the first generation round <ref:2607. p. 4
Where the paper says it

For each evidence type, the table specifies how the setup (S), target action (A), and oracle shape (Q) are constructed. Structural objectives are initialized before the first generation round.

03194#pg6> Most structural objectives are derived through static analysis, including StateSetup, DispatchArm, ExceptionPath, Construction, BoundaryInput, and EntryRoute <ref:2607. p. 4
Where the paper says it

Structural objectives are initialized before the first generation round. Most structural objectives are derived through static analysis, including StateSetup, DispatchArm, ExceptionPath, Construction, BoundaryInput, and EntryRoute.

03194#pg6> However, some testing requirements cannot be reliably inferred through static analysis alone <ref:2607. p. 4
Where the paper says it

Most structural objectives are derived through static analysis, including StateSetup, DispatchArm, ExceptionPath, Construction, BoundaryInput, and EntryRoute. However, some testing requirements cannot be reliably inferred through static analysis alone.

03194#pg6> Therefore TATG performs additional LLM-based objective inference <ref:2607. p. 4
Where the paper says it

However, some testing requirements cannot be reliably inferred through static analysis alone. Therefore, TATG performs additional LLM-based objective inference, using the objective definitions and the focal method source code to identify higher-level testing objectives.

03194#pg6> Feedback-driven objectives are generated incrementally throughout iterative generation through compilation/execution diagnostics, coverage reports, and mutation feedback <ref:2607. p. 4
Where the paper says it

Feedback-driven objectives are generated incrementally throughout iterative generation. Compilation and execution diagnostics, along with coverage reports, generate ValidationRepair objectives, weak assertions identified in generated tests generate OracleStrengthening objectives, and mutation testing generates MutationReachability and MutationAssertion objectives for uncovered and live mutants, respectively.

03194#pg6>

--- Page 5 ---

Figure 1. Overview of TATG. p. 2

Where the paper says it

APPROACH A. Overview TATG is a two-stage, objective-guided framework for LLMbased unit test generation.

TATG addresses this example through two-stage objectiveguided generation <ref:2607. p. 2

Where the paper says it

As a result, a passing test suite may still leave deep paths uncovered, while subsequent repair or regeneration lacks an explicit objective that identifies the unresolved branch or exception behavior requiring attention, such as a specific token-dispatch branch or an exception-translation path. TATG addresses this example through two-stage objectiveguided generation.

03194#pg6> Before generation, static analysis translates branch-relevant facts into explicit objectives <ref:2607. p. 2
Where the paper says it

TATG addresses this example through two-stage objectiveguided generation. Before generation, static analysis translates branch-relevant facts into explicit objectives, such as “exercise the branch where currToken == Token.

03194#pg6> The structural stage first uses compilation and execution diagnostics, together with coverage feedback to improve target reachability and code coverage <ref:2607. p. 2
Where the paper says it

advance() to throw StreamException and assert the resulting IOException”. The structural stage first uses compilation and execution diagnostics, together with coverage feedback, to improve target reachability and code coverage.

03194#pg6> After each round, TATG updates the objective state to record which objectives were satisfied, which failed compilation/execution, and which remain open <ref:2607. p. 2
Where the paper says it

The structural stage first uses compilation and execution diagnostics, together with coverage feedback, to improve target reachability and code coverage. After each round, TATG updates the objective state to record which objectives were satisfied, which failed compilation/execution, and which remain open.

03194#pg6> Only after sufficient structural progress does mutation feedback become actionable <ref:2607. p. 2
Where the paper says it

After each round, TATG updates the objective state to record which objectives were satisfied, which failed compilation/execution, and which remain open. Only after sufficient structural progress does mutation feedback become actionable: mutants in unreached code mainly indicate missing reachability, whereas live mutants in reached code may indicate either insufficient test inputs or weak assertions.

03194#pg6>

--- Page 6 ---

TABLE I

PROJECTION FROM SOURCE-GROUNDED EVIDENCE TO OBJECTIVES.

Category cat Evidence E from static analysis or dynamic feedback Required setup S Target action A Oracle shape Q <ref:2607.03194#pg6> StateSetup Assignment, update, or predicate over a receiver field, e. p. 5

Where the paper says it

TABLE I PROJECTION FROM SOURCE-GROUNDED EVIDENCE TO OBJECTIVES. Category cat Evidence E from static analysis or dynamic feedback Required setup S Target action A Oracle shape Q StateSetup Assignment, update, or predicate over a receiver field, e.

g., this.index++, token = p. p. 5
Where the paper says it

Category cat Evidence E from static analysis or dynamic feedback Required setup S Target action A Oracle shape Q StateSetup Assignment, update, or predicate over a receiver field, e. index++, _token = p.

nextToken, if (this. p. 5
Where the paper says it

index++, _token = p. nextToken(), if (this.

index >= end), or a method call whose receiver resolves to a receiver field, e. p. 5
Where the paper says it

nextToken(), if (this. index >= end), or a method call whose receiver resolves to a receiver field, e.

g., this.context.inObject <ref:2607. p. 0
Where the paper says it

Building on this objective representation, TATG employs a two-stage generation strategy. The structural stage focuses on producing executable tests, improving reachability, and increasing line/branch coverage using static objectives, along arXiv:2607.

03194#pg6> Establish the required receiver or collaborator state through visible constructors, setters, builders, factories, or public methods <ref:2607. p. 5
Where the paper says it

index >= end), or a method call whose receiver resolves to a receiver field, e. inObject() Establish the required receiver or collaborator state through visible constructors, setters, builders, factories, or public methods Invoke the focal method after the required state has been established Assert return value, state transition, emitted output, collaborator interaction, or exception DispatchArm Branch or switch statement whose predicate/selector depends on a concrete code element: if (token == VALUE_STRING), switch (mode), case STRICT:, "json".

03194#pg6> Invoke the focal method after the required state has been established <ref:2607. p. 5
Where the paper says it

index >= end), or a method call whose receiver resolves to a receiver field, e. inObject() Establish the required receiver or collaborator state through visible constructors, setters, builders, factories, or public methods Invoke the focal method after the required state has been established Assert return value, state transition, emitted output, collaborator interaction, or exception DispatchArm Branch or switch statement whose predicate/selector depends on a concrete code element: if (token == VALUE_STRING), switch (mode), case STRICT:, "json".

03194#pg6> Assert return value, state transition, emitted output, collaborator interaction, or exception <ref:2607. p. 5
Where the paper says it

index >= end), or a method call whose receiver resolves to a receiver field, e. inObject() Establish the required receiver or collaborator state through visible constructors, setters, builders, factories, or public methods Invoke the focal method after the required state has been established Assert return value, state transition, emitted output, collaborator interaction, or exception DispatchArm Branch or switch statement whose predicate/selector depends on a concrete code element: if (token == VALUE_STRING), switch (mode), case STRICT:, "json".

03194#pg6> DispatchArm Branch or switch statement whose predicate/selector depends on a concrete code element: if (token == VALUE STRING), switch (mode), case STRICT:, json.equals(format), type == 1, or x instanceof Foo <ref:2607. p. 5
Where the paper says it

inObject() Establish the required receiver or collaborator state through visible constructors, setters, builders, factories, or public methods Invoke the focal method after the required state has been established Assert return value, state transition, emitted output, collaborator interaction, or exception DispatchArm Branch or switch statement whose predicate/selector depends on a concrete code element: if (token == VALUE_STRING), switch (mode), case STRICT:, "json". equals(format), type == 1, or x instanceof Foo Choose argument values, receiver state, or concrete object types that satisfy the selected predicate, match the selected label, or satisfy the selected type test.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Unified objective representation enables explicit tracking of testing requirements which remain unresolved throughout generation, allowing subsequent iterations to focus exclusively on unresolved testing goals rather than repeatedly targeting already-satisfied requirements.

  2. Two-stage objective guided generation coordinates structural coverage improvement and mutation-based oracle strengthening, ensuring that the structural stage focuses on producing executable tests, improving reachability, and increasing line/branch coverage before hardening rounds strengthen assertions using mutation feedback to improve fault detection capability.

  3. The system can perform two distinct types of objective inference: static analysis generates structural objectives for reachability and branch coverage, while LLM-based inference identifies higher-level testing objectives such as ExceptionPath objective corresponding to the try–catch–rethrow path.

  4. Feedback-driven objectives are generated incrementally through iterations, unifying compilation/execution diagnostics, coverage reports, and mutation results into a single objective rather than handling them independently, which allows TATG to reason explicitly about which requirements have been satisfied and which remain unresolved.

  5. The system can generate specific feedback objectives like OracleStrengthening or MutationAssertion based on evidence such as a weak assertion identified in generated tests or a live mutant reported on a source line that was not executed, respectively.

  6. The budget allocation is dynamic, computing the number of requested tests as Br = clip(Bmin, Bmax, ⌈λAr⌉), which allows TATG to allocate resources per active objective rather than using a fixed number of tests for every round.

  7. The system can generate highly specific setup instructions from static analysis objectives, such as configure stream.advance to throw StreamException and assert the resulting IOException, directly translating abstract requirements into actionable test configurations in the prompt.

Abstract

Complex Java methods challenge automated unit test generation, as achieving high coverage and fault detection requires satisfying intricate testing requirements involving program states, execution paths, and assertions. Existing LLM-based approaches leverage program context, static analysis, and dynamic feedback but lack explicit tracking of individual testing requirements across iterations, potentially leading to redundant generation and unresolved testing objectives. We present TATG, a tracking-aware LLM-based unit test generation framework that unifies structural coverage and mutation-based fault detection. TATG introduces a structured objective representation capturing testing requirements derived from static analysis and dynamic feedback, enabling fine-grained tracking of their satisfaction status. It employs a two-stage workflow that first improves structural coverage and subsequently strengthens assertions through mutation-guided hardening. We evaluate TATG on 141 complex Java methods against state-of-the-art baselines, including KTester and PANTA. Results show average improvements of 22.15, 20.14, and 37.66 percentage points in line coverage, branch coverage, and mutation score, respectively. TATG also outperforms general-purpose coding assistants, including Codex and Claude Code, under the same open-weight model configurations, achieving higher average coverage and mutation scores while generating fewer tests and consuming fewer tokens. These findings highlight the value of explicitly tracking testing objectives for effective LLM-based test generation.

Sources

Related papers