On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model".
Jane: Here's the gist of the paper. Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question; Recent advances in language models trained on code have shown superior performance in generating these representations compared to language models trained solely on natural language text <ref:2301.12868#pg2>.
Tom: The large language model of code is vulnerable to carefully crafted adversarial examples <ref:2301.12868#pg3>.
Jane: To overcome this challenge, the paper proposes methods for enhancing robustness without requiring substantial amounts of labelled data or intensive computational resources <ref:2301.12868#pg5>.
Tom: Adversarial Vulnerability Assessment The study evaluates prompt-based semantic parsers based on CODEX, a state-of-the-art language model trained on code <ref:2301.12868#pg6>.
Paper summary: Tom: So, to recap, the paper "On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex" is setting out to find out if we can make prompt-based semantic parsers more robust by just changing how we put examples into the prompt during in-context learning.
Jane: And they're also checking if feeding those models a few examples that are specifically adversarial can actually help improve Codex's ability to handle tricky inputs.
Lu: The study explores three main things: first, whether adding extra examples to the prompt makes the parsers more robust, second, whether injecting few-shot adversarial examples into the prompts helps Codex, and third, if using smarter ways than just random sampling can select better examples for robustness.
Meng: That sounds like a practical set of questions because we constantly deal with noisy inputs in production. So what are they testing these things against?
Lalam: They're probing Codex on two specific semantic parsing benchmarks called GeoQuery and Scholar, which are used to test how well the model handles different kinds of natural language questions.
Tom: And they found that prompt-based semantic parsers are vulnerable to adversarial examples, especially those that come from sentence-level perturbations, and interestingly, Codex seems more sensitive to those sentence-level changes than it is to word-level ones.
Jane: That's a key distinction because it tells us where the weaknesses lie in how these models interpret complex instructions. It means we need to focus our hardening efforts on those longer linguistic changes.
Lu: They also noted that the rewriting model they trained on human paraphrase pairs actually introduces some of these complex linguistic phenomena when testing robustness, which is an interesting finding about how paraphrasing affects stability.
Conclusion: Tom: So, looking at the paper "On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex", the main point is that we can study and improve prompt-based semantic parsing robustness using in-context learning techniques without needing a huge amount of labeled data.
Jane: It really boils down to finding specific strategies, like using different types of examples or better sampling methods, to make these models handle adversarial inputs better when they are used in real applications.
Lu: The authors are showing that by focusing on sentence-level perturbations and testing those against Codex, they get an empirical look at how these large language models behave under pressure.
Meng: For deployment, this suggests that we don't necessarily need a massive adversarial training pipeline; instead, smart prompt design can offer a real boost in reliability without the immense computational cost of traditional adversarial training.
Lalam: The implication for people using these tools is that the way you structure your prompts and the examples you provide matters significantly for how reliable the resulting structured representation will be.
Tom: So, it’s less about training a whole new model and more about engineering better interactions with existing large models to make them more resilient.
Jane: Exactly. It means focusing our attention on those prompt-based methods rather than just scaling up the raw model size for robustness improvements, which is much more efficient.
Terry Yue Zhuo, Zhuang Li, Yujin Huang, Fatemeh Shiri, Weiqing Wang, Gholamreza Haffari, Yuan-Fang Li
CSIRO's Data61, Australia · Monash University, Australia
cs.CL
Submitted: 2023-01-30
Updated: 2026-11-13
Comments: Accepted at EACL2023 (main)
Code: https://github.com/kingoflolz/mesh-transformer-jax
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 74/100
The gist: Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question…
Key concepts
- Semantic Parsing
- This technique aims to turn natural language questions into a structured format that a computer can understand. It's like translating human language into a formal, machine-readable structure so AI models can process the meaning accurately.
- Adversarial Examples
- These are carefully crafted inputs designed specifically to trick an AI model into producing incorrect or undesirable outputs. In this study, researchers tested how sensitive prompt-based parsers are to these malicious inputs.
- In-Context Learning
- This is a method where you improve a language model's performance by providing examples directly within the text of the prompt itself. The paper explored different ways to use more examples or specific types of examples to make the parser more robust against attacks.
Terminology
Summary
Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question; Recent advances in language models trained on code have shown superior performance in generating these representations compared to language models trained solely on natural language text <ref:2301. p. 0 edu
Abstract
Semantic parsing is a technique aimed at constructing a structured representation of the
meaning of a natural-language question. Recent advances in language models trained on
code have shown superior performance in generating these representations compared to language models trained solely on natural language text. This paper presents the first empirical study on
the adversarial robustness of a prompt-based
semantic parser based on CODEX, a stateof-the-art (SOTA) language model trained on
code. Our results demonstrate that the large
language model of code is vulnerable to carefully crafted adversarial examples. Our results demonstrate that the large
language model of code is vulnerable to carefully crafted adversarial examples. To overcome this challenge, we propose methods for
enhancing robustness without requiring substantial amounts of labelled data or intensive
computational resources.Where the paper says it
Where the paper says it
Where the paper says it
Adversarial Vulnerability Assessment
The study evaluates prompt-based semantic parsers based on CODEX, a state-of-the-art language model trained on code <ref:2301.12868#pg6>. The research finds that prompt-based semantic parsers are vulnerable to adversarial examples, particularly those crafted by sentence-level perturbations <ref:2301. p. 1 In this work, we perform a series of experiments to probe CODEX, a large pre-trained model
trained on code, on two semantic parsing benchmarks, GeoQuery (Zelle and Mooney, 1996) and
Scholar (Iyer et al. Our key findings
from the above experiments are as follows:
• Prompt-based semantic parsers are vulnerable
to adversarial examples, particularly the ones
crafted by sentence-level perturbations. The zero-shot parsing performances of
CODEX are shown in Table 2. Our first observation is that CODEX is more vulnerable to sentencelevel perturbations than to word-level perturbations,
as indicated by the more significant performance
gaps between standard and perturbed accuracies on
the sentence-level perturbed test sets. , coreference resolution, numerical reasoning,
negation). We observe that the rewriting model
trained on human paraphrase pairs also introduces
such complex linguistic phenomenons.Where the paper says it
Where the paper says it
Where the paper says it
In-Context Learning Strategies
The paper investigates how different in-context learning methods affect parser robustness <ref:2301.12868#pg5>. The three main aspects of the study are:
-
Determining if the inclusion of additional examples within the prompt during in-context learning improves the robustness of promptbased parsers <ref:2301. p. 1
12868#pg5>.Where the paper says it
the study aims to determine if the inclusion of additional examples within the prompt during incontext learning improves the robustness of promptbased parsers. This hypothesis is based on prior research that has demonstrated that the increase in the size of the training data results in an enhancement of robustness in fully-supervised models (Pang et al.
-
Determining if the integration of few-shot adversarial examples within prompts can improve the robustness of Codex <ref:2301. p. 1
12868#pg5>.Where the paper says it
This hypothesis is based on prior research that has demonstrated that the increase in the size of the training data results in an enhancement of robustness in fully-supervised models (Pang et al. The second part of the study aims to determine if the integration of few-shot adversarial examples within prompts can improve the robustness of Codex.
-
Evaluating if sampling methods other than random sampling can select more effective examples that improve the robustness of prompt-based parsers <ref:2301. p. 1
12868#pg5>.Where the paper says it
, 2016; Tramer and Boneh, 2019). Finally, the third part of the study aims to evaluate if sampling methods other than random sampling can select more effective examples that improve the robustness of prompt-based parsers.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Robustness Evaluation for Prompt-based Semantic Parsing can be enhanced by using in-context learning with more demonstration examples to improve indomain robustness of prompt-based parsers, as stated in one key finding of the study. This allows models like CODEX to better handle variations within the target domain without requiring extensive labeled data.
-
A few-shot example selection strategy can be improved by using diversity-based sampling methods, such as Cluster-TF-IDF, which was found to achieve higher lexical diversity and consequently resulted in higher robust and standard accuracy for CODEX. This suggests that selecting examples based on feature similarity rather than random or confidence scores is more effective.
-
The integration of adversarial examples into prompts can be explored through adversarial in-context learning, although the results indicate that this approach
represents a distinct approach from supervised adversarial training in terms of enhancing the robustness of the model.
This suggests that simply augmenting prompts with adversarial examples has alimited impact on the robustness of parsers, in contrast to supervised adversarial training.
Abstract
Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question. Recent advancements in few-shot language models trained on code have demonstrated superior performance in generating these representations compared to traditional unimodal language models, which are trained on downstream tasks. Despite these advancements, existing fine-tuned neural semantic parsers are susceptible to adversarial attacks on natural-language inputs. While it has been established that the robustness of smaller semantic parsers can be enhanced through adversarial training, this approach is not feasible for large language models in real-world scenarios, as it requires both substantial computational resources and expensive human annotation on in-domain semantic parsing data. This paper presents the first empirical study on the adversarial robustness of a large prompt-based language model of code,. Our results demonstrate that the state-of-the-art (SOTA) code-language models are vulnerable to carefully crafted adversarial examples. To address this challenge, we propose methods for improving robustness without the need for significant amounts of labeled data or heavy computational resources.
Sources
- Towards Zero-Shot Frame Semantic Parsing for Domain Scaling
- Evaluating Large Language Models Trained on Code
- Compositional Semantic Parsing with Large Language Models
- Deceiving Google's Perspective API Built for Detecting Toxic Comments
- TextBugger: Generating Adversarial Text Against Real-world Applications
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
- RoBERTa: A Robustly Optimized BERT Pretraining Approach
- Few-Shot Self-Rationalization with Natural Language Prompts
- Adversarial Training Methods for Semi-Supervised Text Classification
- Stress Test Evaluation for Natural Language Inference
- Adversarial Training Can Hurt Generalization
- Evaluating the Text-to-SQL Capabilities of Large Language Models
- BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing
- Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models
- Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data