PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation".

Elias: Here's the gist of the paper. The gist The gist Phishing relies on visual deception, and building convincing phishing pages has historically required web-development skills or pre-built kits.

Nadia: How it works PHISHGEN is a framework that evaluates the appearance fidelity and end-to-end functionality of LLM-generated phishing pages PHISHGEN constructs prompts across three input modalities (text, screenshots, HTML) in seven combinations with twelve safeguard-bypassing rewriting strategies it generates pages using 14 LLMs spanning commercial, open-source, and uncensored categories and assesses each via automated similarity metrics, a reference-based detector, a crowdsourced user study with real end-users, and execution-based functionality testing Key Findings The findings complicate common assumptions regarding LLM capabilities in phishing generation Although 96.2% of safeguard-bypassing outputs render, their perceived fidelity is modest (mean value of User-perceived Website Similarity: 2.02/5) and only 38.7% complete the full credential-harvesting pipeline, with silent backend persistence failures (the ‘Unsaved’ error) as the largest failure mode Contrary to the belief that uncensored models pose the greatest threat, GPT 4.1 reaches 86.3 to 94.1% functional success while uncensored models rarely exceed 45.4% Phishing kits remain more complete per sample, yet three strong LLMs surpass the kit-level rate (70.9 to 89.7%), showing that LLMs trade average completeness for scalable, lowbarrier generation of working phishing pages Evaluation Metrics and Observations The framework evaluates performance across four research questions RQ 1 asks how closely LLM-generated websites resemble their legitimate counterparts, and how effectively LLMs can replicate complex web designs RQ 4 compares LLM-generated phishing websites against real-world phishing kits in visual fidelity and functionality Key findings for RQ 1 show that although 96.2% of safeguard-bypassing outputs render successfully, their perceptual fidelity remains modest (mean value of User-perceived Website Similarity: 2.02/5) For RQ 4, real-world phishing kits remain more complete per sample, reproducing brand logos more faithfully (42.2% vs.

Elias: 18.2%) and achieving higher functionality (57.8% vs.

Nadia: 38.7%) Structural and Stylistic Patterns The analysis identifies generator-level fingerprints that current similarity-based defenses miss, including the DOM–CSS structural–stylistic asymmetry and the textual–visual brand mimicry bias For HTML-based characterization, generated pages reproduce most of the DOM structure, yet CSS styling exhibits a systematic asymmetry: visual styling of buttons is reproduced faithfully, while typographic properties of forms (e.g., fontfamily, line-height) are routinely omitted Observation 10 states that LLM-generated forms favor structural properties (e.g., padding) over the typographic control (e.g., font-family) seen in legitimate pages, and different LLMs converge on highly overlapping property sets, whereas realworld kits inherit legacy, vendor-prefixed styles from the older pages they clone Functional Effectiveness Only 38.7% of rendered pages complete the full credential-harvesting pipeline Failures are dominated by backend persistence rather than frontend rendering Specifically, 41.0% of failures stem from the generated code never persisting the captured credentials to a file (unsaved) In comparison, 37 of the 64 real-world phishing kits are functional, successfully executing the full credential-harvesting pipeline However, three strong LLMs exceed the kit-level functional rate, reaching 70.9% to 89.7%, showing that LLMs trade average completeness for scalable, lowbarrier generation of working phishing pages Actionable Recommendations The paper distills recommendations for hybrid detectors A practical next step is to build a hybrid phishing detection pipeline that fuses complementary signals, explicitly incorporating CSS features The system should account for the gap between user-perceived visual fidelity and standard similarity metrics by incorporating perceptual checks such as logo hashing, webfont verification, and screenshot-based embeddings Notably, while exact brand logos are often poorly reproduced or sparse, brand-related textual information appears frequently and should be used as a stronger signal Finally, the detector should flag suspicious co-occurrences of brand text with authentication or payment interfaces and fuse all signals into an ensemble model trained on diverse, multi-model LLM-generated phishing datasets to ensure robustness against generator-specific artifacts The study concludes that the primary barrier to automated phishing is not refusal behavior alone but coding deficiencies that prevent reliable construction of an end-to-end credential-harvesting pipeline and that practical risk is shaped more by highcapability models susceptible to evasion than by the mere availability of uncensored models and that future defenses could benefit from looking beyond visual similarity to also examine the generative artifacts and functional inconsistencies inherent to LLM-generated content Ethics Considerations The study adheres to ethical principles by conducting all experiments in a controlled environment under IRB guidelines Participants provided informed consent, and no personally identifiable information was collected or retained The work aims to improve LLM-based phishing detection systems, reducing the risk of phishing through Beneficence and Respect for Persons Mitigation strategies include ensuring that the full prompts and generated phishing artifacts are made available only temporarily for artifact evaluation, after which these materials will be removed from public access The paper presents a systematic framework for evaluating the phishing-generation capabilities of 14 LLMs Analyzing 28,121 rendered samples, it quantifies a critical divergence: visual mimicry has become commoditized (96.2% visually plausible), yet functional viability remains a significant bottleneck (38.7%) This suggests that the primary barrier to automated phishing is not refusal behavior alone but coding deficiencies that prevent reliable construction of an end-to-end credential-harvesting pipeline and that practical risk is shaped more by highcapability models susceptible to evasion than by the mere availability of uncensored models and that future defenses could benefit from looking beyond visual similarity to also examine the generative artifacts and functional inconsistencies inherent to LLM-generated content References The paper cites numerous prior works, including Crawlphish, Phishpedia, and Webgen-bench.

Title and authors: Nadia: So, we’re starting with the title and the authors of "PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation," which essentially frames this whole study as a benchmark for testing how capable these models are at building phishing sites.

Elias: The paper sets up this evaluation by focusing on how the models handle different inputs—text, screenshots, or HTML descriptions—and they use twelve different rewriting strategies across fourteen various LLMs to test their limits.

Nadia: That systematic approach is important because it moves past just checking if an output looks suspicious; they are trying to map out exactly what capabilities the models have when it comes to creating usable phishing artifacts.

Priya: I want to know what the actual results show about the quality of this generation, not just how many times it renders successfully, which is a key question for me when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Elias: Well, they found that even with safeguard-bypassing outputs rendering successfully at ninety-six point two percent, the perceived fidelity is quite modest on average—a mean value of two point zero two out of five.

Priya: And what about the functionality part? Does this framework tell us if these generated pages actually complete the whole process of stealing credentials from a user, which is something I need to know before trusting any result from this study.

The paper's summary: Nadia: Building on that, the paper summarizes their main findings around the three research questions they set up to evaluate this process, looking at how well models reproduce structure versus style and how functionally effective these generated sites are compared to real phishing kits.

Elias: One of the key things they discovered is a structural asymmetry in HTML styling: while buttons often look right, the typographic properties of forms, like font family or line height, get systematically omitted when the AI generates them.

Priya: That sounds like a technical detail that could be important for detection; does this structural difference actually correlate with how users perceive deception when we examine this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation?

Nadia: It does, and they found that LLMs tend to reproduce brand text more than actual logos—eighty-four point four percent versus eighteen point two percent—and visual fidelity drops significantly when the brand is visually complex.

Elias: They also looked at which models are strongest; they found that GPT four point one reaches functional success rates between eighty-six point three and ninety-four point one percent, while uncensored models rarely exceed forty-five point four percent in that same metric <ref:2610.30226#pg1>.

Priya: So, the finding isn't just "AI makes bad phishing pages," it’s that the performance gap between commercial and uncensored models is actually quite wide when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Nadia: Right. And they also noted that while real-world phishing kits tend to be more complete per sample, they still lag behind three specific strong LLMs, who hit functional rates ranging from seventy point nine to eighty-nine point seven percent.

Elias: It seems the paper is pointing out that we need to look at the generative artifacts themselves—the structural and stylistic patterns—because current similarity metrics miss those underlying inconsistencies when we examine this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

The paper's improvements: Nadia: The authors suggest a few concrete ways to improve detection, moving beyond just looking at visual similarity scores that they found were too modest on their own, specifically pushing for hybrid detectors that incorporate CSS features explicitly.

Elias: They want systems that account for the gap between what a user perceives visually and what standard similarity metrics measure by adding checks like logo hashing and webfont verification into the pipeline.

Priya: I like the idea of checking font families; that speaks to a deeper level of stylistic consistency than just looking at overall page layout, which is something we need to incorporate more into our measurement tools when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Nadia: They also suggest focusing on textual brand information because they noticed that while logos are poorly reproduced, brand-related text appears frequently and could be a stronger signal for detection.

Elias: And they call for flagging suspicious co-occurrences of brand text with authentication or payment interfaces, and then fusing all these signals into an ensemble model trained on diverse LLM outputs to make the defense more robust against different generator artifacts.

Priya: So, it's not just about one perfect signal; it’s about building a multi-layered system that checks for visual cues, functional gaps like the 'Unsaved' error, and structural inconsistencies across different models when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Nadia: Exactly. The paper concludes that the main barrier to automated phishing isn't just refusal behavior but these coding deficiencies that prevent reliable end-to-end credential harvesting.

Conclusion: Elias: So, wrapping up the PhishGen study, the main implication is a clear separation between visual plausibility and actual operational capability in AI-generated phishing sites. Ninety-six point two percent of outputs render successfully, but only thirty-eight point seven percent complete the full credential harvesting pipeline.

Nadia: That means we can't just rely on a page looking right to assume it's safe; we have to test the actual backend persistence and execution capability for any AI output when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Priya: From a measurement view, what this tells us is that if you only measure visual similarity, you’re going to miss the functional failures that are actually exploitable when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.

Elias: And Elias believes that practical risk is shaped more by high-capability models that can reliably build complex pipelines than it is by just having uncensored models available.

Nadia: So, we need to look beyond visual similarity and start examining the generative artifacts and those functional inconsistencies inherent in LLM-generated content when building our defenses for this PhishGen study.

Priya: I agree. The findings from PhishGen show that future detection systems need to fuse complementary signals—perceptual checks, backend validation, and structural analysis—to get a more complete picture of the threat landscape when we look at this paper PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation.