On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex

arXiv:2301.12868 · cs.CL · Submitted 2023-01-30 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model".

Jane: Here's the gist of the paper. Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question; Recent advances in language models trained on code have shown superior performance in generating these representations compared to language models trained solely on natural language text <ref:2301.12868#pg2>.

Tom: The large language model of code is vulnerable to carefully crafted adversarial examples <ref:2301.12868#pg3>.

Jane: To overcome this challenge, the paper proposes methods for enhancing robustness without requiring substantial amounts of labelled data or intensive computational resources <ref:2301.12868#pg5>.

Tom: Adversarial Vulnerability Assessment The study evaluates prompt-based semantic parsers based on CODEX, a state-of-the-art language model trained on code <ref:2301.12868#pg6>.

Paper summary: Tom: So, to recap, the paper "On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex" is setting out to find out if we can make prompt-based semantic parsers more robust by just changing how we put examples into the prompt during in-context learning.

Jane: And they're also checking if feeding those models a few examples that are specifically adversarial can actually help improve Codex's ability to handle tricky inputs.

Lu: The study explores three main things: first, whether adding extra examples to the prompt makes the parsers more robust, second, whether injecting few-shot adversarial examples into the prompts helps Codex, and third, if using smarter ways than just random sampling can select better examples for robustness.

Meng: That sounds like a practical set of questions because we constantly deal with noisy inputs in production. So what are they testing these things against?

Lalam: They're probing Codex on two specific semantic parsing benchmarks called GeoQuery and Scholar, which are used to test how well the model handles different kinds of natural language questions.

Tom: And they found that prompt-based semantic parsers are vulnerable to adversarial examples, especially those that come from sentence-level perturbations, and interestingly, Codex seems more sensitive to those sentence-level changes than it is to word-level ones.

Jane: That's a key distinction because it tells us where the weaknesses lie in how these models interpret complex instructions. It means we need to focus our hardening efforts on those longer linguistic changes.

Lu: They also noted that the rewriting model they trained on human paraphrase pairs actually introduces some of these complex linguistic phenomena when testing robustness, which is an interesting finding about how paraphrasing affects stability.

Conclusion: Tom: So, looking at the paper "On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex", the main point is that we can study and improve prompt-based semantic parsing robustness using in-context learning techniques without needing a huge amount of labeled data.

Jane: It really boils down to finding specific strategies, like using different types of examples or better sampling methods, to make these models handle adversarial inputs better when they are used in real applications.

Lu: The authors are showing that by focusing on sentence-level perturbations and testing those against Codex, they get an empirical look at how these large language models behave under pressure.

Meng: For deployment, this suggests that we don't necessarily need a massive adversarial training pipeline; instead, smart prompt design can offer a real boost in reliability without the immense computational cost of traditional adversarial training.

Lalam: The implication for people using these tools is that the way you structure your prompts and the examples you provide matters significantly for how reliable the resulting structured representation will be.

Tom: So, it’s less about training a whole new model and more about engineering better interactions with existing large models to make them more resilient.

Jane: Exactly. It means focusing our attention on those prompt-based methods rather than just scaling up the raw model size for robustness improvements, which is much more efficient.

Terry Yue Zhuo, Zhuang Li, Yujin Huang, Fatemeh Shiri, Weiqing Wang, Gholamreza Haffari, Yuan-Fang Li

CSIRO's Data61, Australia · Monash University, Australia

cs.CL

Submitted: 2023-01-30

Updated: 2026-11-13

Comments: Accepted at EACL2023 (main)

Code: https://github.com/kingoflolz/mesh-transformer-jax

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 74/100

The gist: Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question…

Key concepts

Semantic Parsing
This technique aims to turn natural language questions into a structured format that a computer can understand. It's like translating human language into a formal, machine-readable structure so AI models can process the meaning accurately.
Adversarial Examples
These are carefully crafted inputs designed specifically to trick an AI model into producing incorrect or undesirable outputs. In this study, researchers tested how sensitive prompt-based parsers are to these malicious inputs.
In-Context Learning
This is a method where you improve a language model's performance by providing examples directly within the text of the prompt itself. The paper explored different ways to use more examples or specific types of examples to make the parser more robust against attacks.

Terminology

Summary

Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question; Recent advances in language models trained on code have shown superior performance in generating these representations compared to language models trained solely on natural language text <ref:2301. p. 0

Where the paper says it

edu Abstract Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question. Recent advances in language models trained on code have shown superior performance in generating these representations compared to language models trained solely on natural language text.

12868#pg2>. The large language model of code is vulnerable to carefully crafted adversarial examples <ref:2301. p. 0
Where the paper says it

This paper presents the first empirical study on the adversarial robustness of a prompt-based semantic parser based on CODEX, a stateof-the-art (SOTA) language model trained on code. Our results demonstrate that the large language model of code is vulnerable to carefully crafted adversarial examples.

12868#pg3>. To overcome this challenge, the paper proposes methods for enhancing robustness without requiring substantial amounts of labelled data or intensive computational resources <ref:2301. p. 0
Where the paper says it

Our results demonstrate that the large language model of code is vulnerable to carefully crafted adversarial examples. To overcome this challenge, we propose methods for enhancing robustness without requiring substantial amounts of labelled data or intensive computational resources.

12868#pg5>.

Adversarial Vulnerability Assessment

The study evaluates prompt-based semantic parsers based on CODEX, a state-of-the-art language model trained on code <ref:2301.12868#pg6>. The research finds that prompt-based semantic parsers are vulnerable to adversarial examples, particularly those crafted by sentence-level perturbations <ref:2301. p. 1

Where the paper says it

In this work, we perform a series of experiments to probe CODEX, a large pre-trained model trained on code, on two semantic parsing benchmarks, GeoQuery (Zelle and Mooney, 1996) and Scholar (Iyer et al. Our key findings from the above experiments are as follows: • Prompt-based semantic parsers are vulnerable to adversarial examples, particularly the ones crafted by sentence-level perturbations.

12868#pg7>. Specifically, the results show that CODEX is more vulnerable to sentence-level perturbations than to word-level perturbations <ref:2301. p. 5
Where the paper says it

The zero-shot parsing performances of CODEX are shown in Table 2. Our first observation is that CODEX is more vulnerable to sentencelevel perturbations than to word-level perturbations, as indicated by the more significant performance gaps between standard and perturbed accuracies on the sentence-level perturbed test sets.

12868#pg8>. The authors observe that the rewriting model trained on human paraphrase pairs also introduces such complex linguistic phenomenons <ref:2301. p. 5
Where the paper says it

, coreference resolution, numerical reasoning, negation). We observe that the rewriting model trained on human paraphrase pairs also introduces such complex linguistic phenomenons.

12868#pg8>.

In-Context Learning Strategies

The paper investigates how different in-context learning methods affect parser robustness <ref:2301.12868#pg5>. The three main aspects of the study are:

  1. Determining if the inclusion of additional examples within the prompt during in-context learning improves the robustness of promptbased parsers <ref:2301. p. 1

    Where the paper says it

    the study aims to determine if the inclusion of additional examples within the prompt during incontext learning improves the robustness of promptbased parsers. This hypothesis is based on prior research that has demonstrated that the increase in the size of the training data results in an enhancement of robustness in fully-supervised models (Pang et al.

    12868#pg5>.

  2. Determining if the integration of few-shot adversarial examples within prompts can improve the robustness of Codex <ref:2301. p. 1

    Where the paper says it

    This hypothesis is based on prior research that has demonstrated that the increase in the size of the training data results in an enhancement of robustness in fully-supervised models (Pang et al. The second part of the study aims to determine if the integration of few-shot adversarial examples within prompts can improve the robustness of Codex.

    12868#pg5>.

  3. Evaluating if sampling methods other than random sampling can select more effective examples that improve the robustness of prompt-based parsers <ref:2301. p. 1

    Where the paper says it

    , 2016; Tramer and Boneh, 2019). Finally, the third part of the study aims to evaluate if sampling methods other than random sampling can select more effective examples that improve the robustness of prompt-based parsers.

    12868#pg5>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Robustness Evaluation for Prompt-based Semantic Parsing can be enhanced by using in-context learning with more demonstration examples to improve indomain robustness of prompt-based parsers, as stated in one key finding of the study. This allows models like CODEX to better handle variations within the target domain without requiring extensive labeled data.

  2. A few-shot example selection strategy can be improved by using diversity-based sampling methods, such as Cluster-TF-IDF, which was found to achieve higher lexical diversity and consequently resulted in higher robust and standard accuracy for CODEX. This suggests that selecting examples based on feature similarity rather than random or confidence scores is more effective.

  3. The integration of adversarial examples into prompts can be explored through adversarial in-context learning, although the results indicate that this approach represents a distinct approach from supervised adversarial training in terms of enhancing the robustness of the model. This suggests that simply augmenting prompts with adversarial examples has a limited impact on the robustness of parsers, in contrast to supervised adversarial training.

Abstract

Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question. Recent advancements in few-shot language models trained on code have demonstrated superior performance in generating these representations compared to traditional unimodal language models, which are trained on downstream tasks. Despite these advancements, existing fine-tuned neural semantic parsers are susceptible to adversarial attacks on natural-language inputs. While it has been established that the robustness of smaller semantic parsers can be enhanced through adversarial training, this approach is not feasible for large language models in real-world scenarios, as it requires both substantial computational resources and expensive human annotation on in-domain semantic parsing data. This paper presents the first empirical study on the adversarial robustness of a large prompt-based language model of code,. Our results demonstrate that the state-of-the-art (SOTA) code-language models are vulnerable to carefully crafted adversarial examples. To address this challenge, we propose methods for improving robustness without the need for significant amounts of labeled data or heavy computational resources.

Sources

Related papers