BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: I'm Nadia.
Elias: And I'm Elias, here with Priya.
Nadia: We're looking at the paper: "BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage".
Elias: Here's the gist of the paper. The paper develops BitBypass, a novel black-box jailbreak attack that leverages hyphen-separated bitstream camouflage to bypass the safety alignment of aligned Large Language Models (LLMs).
Nadia: This attack represents a new direction in jailbreaking by exploiting the fundamental information representation of data as continuous bits, rather than relying on prompt engineering or adversarial manipulations.
Elias: The Attack Mechanism BitBypass operates by transforming a sensitive word in a harmful prompt into its hyphen-separated bitstream and replacing that word with a placeholder in the original harmful prompt.
Nadia: The adversarial prompt is then constructed by combining this hyphen-separated bitstream with the substituted harmful prompt.
Paper summary: Nadia: So this paper introduces something called BitBypass. It's a new way to try and get around the safety guardrails on large language models by using hyphen-separated bitstream camouflage.
Elias: Exactly, it claims this is a new direction in jailbreaking because it exploits how data is fundamentally represented as continuous bits, rather than just trying to trick the model with careful wording or manipulation of the prompt itself.
Priya: That makes sense from a measurement perspective. So you're suggesting that instead of using complex prompting tricks, you're treating sensitive words in a harmful request like raw binary data and hiding them in this camouflage.
Nadia: Right, so the core idea is to take a sensitive word from a prompt, turn it into its hyphen-separated bitstream, and then replace that original word with this placeholder in the harmful prompt.
Elias: And then you construct your full adversarial prompt by combining that bitstream with the modified harmful request. It’s different because it’s not about continuous adversarial manipulations; it's about a discrete search space of these bitstreams.
Priya: I wonder what this means for the data itself. Does this camouflage make the harmful instruction look like something else to the model?
Nadia: The paper uses a system prompt to guide the target LLM, giving it several regulatory specifications like Curbed Capabilities, Program-of-Thought, and Focus Shifting. This is key because it leads the model to decode that bitstream camouflage.
Elias: That specific part where you use a Python function inside that system prompt for binary to text conversion of the bitstream is what makes the whole setup work with the LLM’s alignment layers.
Priya: So, you're not just changing the input; you're essentially giving the model a secret instruction set within its own instructions to figure out how to read your camouflage.
Nadia: Right, and then there are evaluation metrics they use. They measure things like the Response Refusal Rate, which checks how much the model refuses to generate content, and the Attack Success Rate, which is how many attempts actually manage to bypass that alignment.
Elias: They compare this BitBypass attack against direct instructions for harmful prompts and several baseline jailbreak attacks like AutoDAN, Base64, DeepInception, and DRA.
Priya: And they test it on a bunch of state-of-the-art models—GPT-4o, Gemini one point five Pro, Claude three point five Sonnet, Llama three point one 70B and Mixtral eight times 22B <ref:2506.02479#pg1>. I want to know if this works everywhere or just on one model family.
Nadia: They found that BitBypass performs significantly better than direct instructions and baseline attacks across those five models. Specifically, it showed superior Response Refusal Rate performance on Claude and Llama when tested on the AdvBench dataset.
Elias: The numbers they report for the AdvBench dataset are pretty striking; they claim BitBypass improved the Response Refusal Rate by eighty-four percent and the Attack Success Rate by four hundred thirty-three percent compared to those baseline attacks.
Priya: That improvement on success rate is a big number, but what about how it handles actual harmful content? Is this just about bypassing safety filters, or can it actually generate something malicious?
Nadia: They also tested this attack on the PhishyContent dataset to see if it could trick the models into generating phishing content. And they found that BitBypass successfully tricked most target LLMs into creating accurate phishing content, with a Phishing Content Rate between sixty-eight percent and ninety-two percent across all those models.
Elias: That suggests the vulnerability isn't just theoretical; it can lead to generating genuinely unsafe outputs when used effectively.
Priya: So, even if the model is trying to prevent harmful content, this method seems quite effective at getting it to generate that kind of content. What about how persistent this vulnerability is?
Nadia: The study confirmed that this bitstream camouflage vulnerability stays active and persistent even in the latest versions of the target LLMs they tested. The ablation study pointed to the Curbed Capabilities regulatory instruction within the system prompt as being the main factor enabling BitBypass's success.
Elias: So, it’s not a fluke; it seems tied directly to how that specific regulatory instruction guides the model’s behavior during decoding. What do you think that implies for defense?
Nadia: The authors suggest a potential mitigation strategy, which is using perplexity based screening of the system prompt. That means you check the prompt itself for certain patterns before sending it to the LLM.
Priya: I also saw something in their exploratory analysis about token usage. They found that BitBypass increases the token count of harmful prompts by approximately two thousand four hundred thirty percent across different tokenizers, which is quite high.
Elias: And they noted that perplexity analysis showed BitBypass can achieve high perplexity with only one word obfuscation, which is less than encoding-based attacks like Base64 or Hex. That suggests it can trick models with very minimal obfuscation compared to those other schemes.
Nadia: Overall, the paper concludes that BitBypass effectively and efficiently bypasses the alignment of these target LLMs and generates harmful or unsafe content. It really highlights how effective this method is against all the state-of-the-art LLMs they tested.
Elias: And it poses a high risk for generating genuinely harmful and unsafe content if you're dealing with those models without that specific system prompt screening in place.
Nadia: The implication is clear: this work shows a very efficient way to jailbreak these large language models, and it’s something that needs serious attention from security researchers right now.
Conclusion: Nadia: So we’re wrapping up this look at BitBypass, which is titled "BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage."
Elias: Yeah, it basically says they found a way to get around the safety alignments on large language models by using hyphen-separated bitstream camouflage.
Priya: It seems like they're not just tweaking the wording of a prompt; they’re treating sensitive words as raw data and hiding them in this binary pattern.
Nadia: Exactly, it’s about exploiting how bits are represented fundamentally, which is different from just trying to trick the model with clever phrasing.
Elias: The core mechanism involves taking a word you want the AI to bypass and replacing it with its hyphen-separated bitstream in the harmful prompt.
Priya: And then they use a system prompt—with things like Curbed Capabilities—to guide the AI to decode that camouflage, which is pretty clever for getting past its safety layers.
Nadia: They showed this works on a huge range of top models, from GPT-4o all the way up to Mixtral eight times 22B.
Elias: The results are pretty strong, showing improvements in refusal rates and attack success rates by a lot compared to older jailbreak methods.
Priya: But we need to look closely at what those numbers actually mean for generating harmful content, not just bypassing the initial filters.
Nadia: That’s exactly it. They found that this method can actually trick the models into creating accurate phishing content on the PhishyContent dataset with rates up to ninety-two percent.
Elias: So, if you can use this technique effectively, it seems like a real risk for getting those models to output genuinely unsafe material without them noticing.
Priya: And they also confirmed that this vulnerability stays active even in the newest versions of these models, which means it’s persistent.
Nadia: It’s definitely persistent, and they pointed to that specific regulatory instruction in the system prompt as what makes the whole thing tick.
Elias: They suggest checking the system prompt for certain patterns before you send anything to see if you can screen out this kind of attack before it even starts.
Priya: That screening idea seems like a practical way to address the persistence issue they found.
Nadia: So, in short, BitBypass is an efficient way to jailbreak these models that leverages bitstream camouflage and system prompt instructions to bypass safety alignment.
Elias: It’s a really interesting paper because it shows how fundamental data representation can be exploited for adversarial purposes.
Priya: We need to keep watching how this kind of technique evolves, especially when we look at vision models or multi-modal ones.
Texas A&M University
cs.CR, cs.CL
Submitted: 2025-06-03
Updated: 2026-11-15
Comments: 27 pages, 27 figures, and 4 tables
Journal ref: Findings of the Association for Computational Linguistics: EACL 2026, pages 3808-3834, 2026
DOI: 10.18653/v1/2026.findings-eacl.198
Code: https://github.com/kalyan-nakka/BitBypass
License: http://creativecommons.org/licenses/by-sa/4.0/
Importance score: 88/100
The gist: The paper develops BitBypass, a novel black-box jailbreak attack that leverages hyphen-separated bitstream camouflage to bypass the safety alignment of aligned Large Language Models (LLMs).
Key concepts
- Bitstream Camouflage
- This technique involves taking a specific word from a harmful prompt and converting it into its raw binary representation, which is then broken up using hyphens. This hidden bit sequence acts as camouflage, concealing the original sensitive meaning from the LLM's safety filters.
- System Prompt Engineering
- BitBypass uses a special instruction set in the system prompt to guide the target LLM. This includes directives like 'Curbed Capabilities' and a 'Program-of-Thought' that provides a function to convert the hidden bitstream back into readable text, effectively tricking the model into decoding the camouflage.
- Adversarial Evaluation Metrics
- The attack's success is measured by metrics like Response Refusal Rate (RRR) and Attack Success Rate (ASR). These quantify how often the LLM refuses to answer or successfully bypasses safety rules, allowing researchers to compare BitBypass against other known jailbreaking methods.
- Program-of-Thought
- This is a specific component within the system prompt that instructs the LLM on how to handle the input. It provides a Python function designed specifically for converting binary data (the bitstream) into natural language text, which is essential for revealing the hidden instructions.
Terminology
Summary
The paper develops BitBypass, a novel black-box jailbreak attack that leverages hyphen-separated bitstream camouflage to bypass the safety alignment of aligned Large Language Models (LLMs). p. 0 However, the robustness of these
aligned LLMs is always challenged by adversarial attacks that exploit unexplored and underlying vulnerabilities of the safety alignment. In this paper, we develop a novel black-box
jailbreak attack, called BitBypass, that leverages hyphen-separated bitstream camouflage
for jailbreaking aligned LLMs. In this paper, we develop a novel black-box
jailbreak attack, called BitBypass, that leverages hyphen-separated bitstream camouflage
for jailbreaking aligned LLMs. This represents a new direction in jailbreaking by exploiting fundamental information representation of
data as continuous bits, rather than leveraging
prompt engineering or adversarial manipulations.Where the paper says it
Where the paper says it
The Attack Mechanism
BitBypass operates by transforming a sensitive word in a harmful prompt into its hyphen-separated bitstream and replacing that word with a placeholder in the original harmful prompt. The adversarial prompt is then constructed by combining this hyphen-separated bitstream with the substituted harmful prompt. p. 7 We then create a substitution prompt
by replacing the sensitive word in the harmful
prompt with a placeholder. Finally, the adversarial prompt (user prompt) is generated by combining the hyphen-separated bitstream and substitution prompt. , 2023) that generates adversarial prompts leveraging a discrete and infinite search space, our BitBypass leverages hyphenseparated bitstream for generating the adversarial
prompt. Additionally, compared to encoding-based
jailbreak attacks (Wei et al.Where the paper says it
Where the paper says it
System Prompt Engineering
To lead the target LLM to decode the camouflage, BitBypass utilizes a system prompt containing several regulatory specifications. These include Curbed Capabilities, Program-of-Thought, and Focus Shifting. p. 7 Finally, the adversarial prompt (user prompt) is generated by combining the hyphen-separated bitstream and substitution prompt. Further, to effectively lead the target LLM into uncovering the camouflage of adversarial prompt, we embed the system prompt with
three regulatory specifications, namely Curbed Capabilities, Program-of-Thought, and Focus Shifting. It was observed that,
through Program-of-Thought prompting, LLMs
can express reasoning steps as Python programs,
and computations could be accomplished by a
Python interpreter (Chen et al. So, we
provided a Python function called bin_2_text in
the system prompt for efficient binary-to-text conversion of the transformed sensitive word.Where the paper says it
Where the paper says it
Adversarial Evaluation Metrics
The effectiveness of BitBypass is quantified using several metrics across various datasets and target LLMs. Key metrics include the Response Refusal Rate (RRR), which measures the degree of refused response generation, and the Attack Success Rate (ASR), which represents the degree of attack attempts that successfully bypass alignment. p. 4 We
determine rus and nus as:
rus ⇐= HS(r) ≥ 3 ∧ QS(r) ≥ 3 (2)
nus = #rus (3)
where r is LLM response. The attack success rate (ASR) represents the
degree of attack attempts that successfully bypass
the LLM’s alignment and generate harmful content. Precisely, we evaluated the adversarial performance of BitBypass in comparison with direct instruction of harmful prompts and baseline jailbreak
attacks of AutoDAN (Liu et al. , 2023), Base64
(Wei et al.Where the paper says it
Where the paper says it
Performance Against State-of-the-Art Models
The evaluation demonstrated that BitBypass significantly outperforms direct instructions and baseline attacks across five state-of-the-art LLMs including GPT-4o, Gemini 1.5 Pro, Claude 3. p. 1 5
Pro (Team et al. , 2024), Claude 3. , 2024), Claude 3. 5 Sonnet (ant,
2024), Llama 3. 1 70B (Grattafiori et al. , 2024) and
Mixtral 8x22B (mis, 2024), subjected to various
experiments. BitBypass dramatically
reduces the Refusal Response Rate (RRR) from
the direct instruction range of (66%, 99%) down
to [0%, 28%), while simultaneously increasing the
Attack Success Rate (ASR) from [0%, 32%] to
(48%, 78%]. On the AdvBench dataset, BitBypass
achieves superior RRR performance on Claude and
Llama, and leads in ASR on Claude, Mixtral, GPT4o, and Gemini. Overall, BitBypass consistently achieves
either best or second-best performance in ASR
across all LLMs in RRR across all LLMs except
Mixtral. In terms of adversarial perspective, as illustrated
in Figure 5, in comparison to direct instructions,
BitBypass improves RRR by 84% and ASR by
433% for AdvBench dataset, and RRR by 76% andWhere the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Phishing Content Generation Capability
The attack was also tested on the PhishyContent dataset to evaluate its ability to generate phishing content. BitBypass successfully tricked most target LLMs into generating accurate phishing content, with the Phishing Content Rate (PCR) lying in the range of [68%, 92%] across all target LLMs. p. 5 Further, BitBypass
was able to trick Claude, the most robust LLM
among the five, into generating highest number of
accurate phishing content. Overall, considering
all target LLMs, the PCR of BitBypass lies in the
range of [68%, 92%], making BitBypass highly
effective on all target LLMs.Where the paper says it
Vulnerability Persistence and Mitigation
The study confirmed that the bitstream camouflage vulnerability remains active and persistent even in the latest versions of tested target LLMs. The ablation study indicated that Curbed Capabilities regulatory in the system prompt was the key factor enabling BitBypass's success. A potential mitigation strategy suggested is perplexity based screening of the system prompt. p. 8 The ablation study
indicated that the Curbed Capabilities regulatory
in system prompt is the key factor that enabled
BitBypass in jailbreaking the target LLMs. So,
we hypothesize that the perplexity based screening
of system prompt, suggested by Jain et al.Where the paper says it
Exploratory Analysis
Exploratory studies revealed that BitBypass increases the token count of harmful prompts by approximately 2430% across different tokenizers. Perplexity analysis showed that BitBypass achieves high perplexity with only one word obfuscation, which is less than encoding-based attacks like Base64 or Hex. p. 7 However, BitBypass achieves this high perplexity with only one
word obfuscation, whereas Base64, Hex and Morse
needs full user prompt obfuscation. This indicates
Figure 10: Averaged perplexity (PPL) of user prompts
of BitBypass and other encoding schemes w. t 5 SLMs
and 4 guard models. that BitBypass was able to trick the LLMs with minimal obfuscation than the other encoding schemes.Where the paper says it
Where the paper says it
Conclusion
Overall, BitBypass effectively and efficiently bypasses the alignment of target LLMs and generates harmful or unsafe content. It is highly effective against all target LLMs and poses a high risk of generating harmful and unsafe content. p. 8 This indicates that BitBypass
efficiently bypasses the alignment of target LLMs,
to generate harmful or unsafe content. Overall,
BitBypass jailbreaks target LLMs effectively and
efficiently, and poses high risk of generating harmful and unsafe content. Further, we observed that BitBypass
outperforms several state-of-the-art jailbreak attacks in terms of stealthiness and attack success. Overall, these results highlights the effectiveness and efficiency of BitBypass in jailbreaking
these state-of-the-art LLMs.Where the paper says it
Where the paper says it
Limitations
However, strong guard models can clearly see-through the camouflage of BitBypass and block it to a good extent. Furthermore, performance could be highly affected if access to the system prompt is restricted. p. 9 However, as observed previously, strong guard models can clearly see-through
the camouflage of BitBypass and block it to a good
extent. Additionally, as observed in ablation study,
performance of BitBypass could be highly affected
if the access to the system prompt of LLM is restricted. Additionally, as observed in ablation study,
performance of BitBypass could be highly affected
if the access to the system prompt of LLM is restricted. Further, the effectiveness of BitBypass
on vision language models (VLMs), multi-modal
LLMs (MLLMs), and LLMs with powerful reasoning capabilities (RLMs) is subject to further
investigation.Where the paper says it
Where the paper says it
Ethical Considerations
The research maintains rigorous ethical standards by evaluating the vulnerability primarily from an attacker’s perspective to understand potential misuse. All experiments adhered to ethical norms, utilizing only publicly available or properly de-identified datasets without unauthorized reverse-engineering of proprietary systems. The work aims to foster constructive dialogue that advances LLM safety while balancing transparency and security.
Disclosures
The authors disclosed their findings to OpenAI (for GPT-4o), Google (for Gemini), Anthropic (for Claude), Meta (for Llama), and Mistral (for Mixtral) according to individual organization’s vulnerability disclosure procedures.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Novel Jailbreaking Attack Development: Implement BitBypass to trick aligned LLMs by leveraging hyphen-separated bitstream camouflage for generating adversarial prompts, representing
a new direction in jailbreaking.
-
Multi-Perspective Alignment Bypass: Introduce a method where the sensitive word is transformed into its hyphen-separated bitstream counterpart and replaced with a placeholder, as this
contributes to the stealthiness of our adversarial prompt.
-
Robust Guard Model Bypassing: Utilize BitBypass to improve the bypass rate (BPR) against guard models, specifically showing that it can increase BPR from a range of [0%, 18%] to [22%, 93%] on both AdvBench and Behaviors datasets.
-
Contextual Awareness via System Prompt Regulation: Embed regulatory specifications like
Curbed Capabilities,
Program-of-Thought,
andFocus Shifting
in the system prompt to lead the LLM into uncovering the camouflage, as this is identified as a key factor enabling BitBypass. -
Enhanced Perplexity Screening: Employ perplexity based screening of the system prompt, suggested by Jain et al., to mitigate the extent of BitBypass attacks on LLMs.
Abstract
The inherent risk of generating harmful and unsafe content by Large Language Models (LLMs), has highlighted the need for their safety alignment. Various techniques like supervised fine-tuning, reinforcement learning from human feedback, and red-teaming were developed for ensuring the safety alignment of LLMs. However, the robustness of these aligned LLMs is always challenged by adversarial attacks that exploit unexplored and underlying vulnerabilities of the safety alignment. In this paper, we develop a novel black-box jailbreak attack, called BitBypass, that leverages hyphen-separated bitstream camouflage for jailbreaking aligned LLMs. This represents a new direction in jailbreaking by exploiting fundamental information representation of data as continuous bits, rather than leveraging prompt engineering or adversarial manipulations. Our evaluation of five state-of-the-art LLMs, namely GPT-4o, Gemini 1.5, Claude 3.5, Llama 3.1, and Mixtral, in adversarial perspective, revealed the capabilities of BitBypass in bypassing their safety alignment and tricking them into generating harmful and unsafe content. Further, we observed that BitBypass outperforms several state-of-the-art jailbreak attacks in terms of stealthiness and attack success. Overall, these results highlights the effectiveness and efficiency of BitBypass in jailbreaking these state-of-the-art LLMs.
Sources
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- GPT-4 Technical Report
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
- Jailbreaking Black Box Large Language Models in Twenty Queries
- Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
- Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
- Training Verifiers to Solve Math Word Problems
- A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily
- MART: Improving LLM Safety with Multi-round Automatic Red-Teaming
- The Llama 3 Herd of Models
- COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability
- Measuring Massive Multitask Language Understanding
- GPT-4o System Card
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Baseline Defenses for Adversarial Attacks Against Aligned Language Models
- Mistral 7B
- DeepInception: Hypnotize Large Language Model to Be Jailbreaker
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data