SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code
summary
The gist
Scientific figures are the interface through which research claims are inspected and reused, but final published panels rarely expose the data or plotting code that produced them The paper introduces…
In short
SCIFIGURE2CODE creates a benchmark to test AI's ability to reconstruct scientific figures into editable Python code. It transforms published panels into reference packages and evaluates models on generating executable plotting logic that preserves layout, not just visual similarity. This reveals current general-purpose vision models struggle with complex scientific panel structures.
Key concepts
- SCIFIGURE2CODE
- An AI-reconstructed benchmark designed to evaluate presentation recovery. It turns final published scientific panels into auditable reference packages containing editable Python programs, testing if an AI can recreate the figure's arrangement and read structure.
- Presentation Recovery
- The core task being evaluated: generating functional, editable Python code that accurately reproduces the layout of a scientific panel. This tests an AI's understanding of visual hierarchy, encodings (like axes and legends), and annotations rather than just recognizing the image.
- Image-Only vs. Caption-Assisted
- Two evaluation settings: one where the model only sees the figure (image-only) and another where it receives the image plus metadata like captions or DOIs (caption-assisted). This tests how much context is needed for an AI to successfully reconstruct complex scientific panels.
- Executable Plotting Logic
- The required output format for evaluation. Models must generate actual, runnable Python programs that use standard plotting primitives. Submissions relying on direct image dependencies or raster tracing are invalid; the focus is on generating functional code.
Terminology used across episodes
This episode discusses
- SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code · Paper Radio
- Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots
- RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluation
- ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
- Chart-to-Text: A Large-Scale Benchmark for Chart Summarization
- ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summaries
- ChartLlama: A Multimodal LLM for Chart Understanding and Generation
- ChartBench: A Benchmark for Complex Visual Reasoning in Charts
- Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering
- Evaluating Large Language Models Trained on Code
- Program Synthesis with Large Language Models
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- MMBench: Is Your Multi-modal Model an All-around Player?
- MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
- MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization
- Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
The paper
SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code · Read on arXiv
Tsinghua University
Scientific figures are the interface through which research claims are inspected and reused, but final published panels rarely expose the data or plotting code that produced them. Recovering this hidden provenance from pixels is therefore underdetermined. We introduce SciFigure2Code, an AI-reconstructed benchmark that evaluates presentation recovery: generating editable Python programs that preserve how a scientific panel is arranged and read. Role-specialized Codex agents generate, execute, visually refine, and audit silver-standard presentation programs that capture geometry, visual hierarchy, encodings, annotations, and typography. This reconstruction-and-audit protocol turns final published panels into auditable reference packages; the resulting resource contains 6,740 reviewed panels and SciFigureBench, a balanced 337-panel test set across 31 chart subtypes, five domains, and three complexity levels. Across 14 zero-shot models, execution and structural fidelity remain weak; a 328-panel full-figure study further exposes failures to locate and isolate requested panels. Claude Opus 4.7 leads image-only reconstruction, Claude Opus 4.6 leads caption-assisted reconstruction, and plan-then-code prompting improves all four tested models. SciFigure2Code provides an auditable testbed for constructing editable, visually faithful scientific figure presentations.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code".
Jane: Here's the gist of the paper. Scientific figures are the interface through which research claims are inspected and reused, but final published panels rarely expose the data or plotting code that produced them The paper introduces SCIFIGURE2CODE, an AI-reconstructed benchmark that instead evaluates presentation recovery: generating editable Python programs that preserve how a scientific panel is arranged and read This reconstruction-and-audit protocol turns final published panels into auditable reference packages; the resulting resource contains 6,740 reviewed panels and SCIFIGUREBENCH, a balanced 337-panel test set across 31 chart subtypes, five domains, and three complexity levels Across 14 zero-shot models in image-only and caption-assisted settings, execution, multi-component layouts, axes, legends, and scientific labels remain weak Benchmark Construction The SCIFIGURE2CODE benchmark is constructed by converting final published panels into auditable presentation-reference packages through a Codex-driven workflow This pipeline involves several stages, including acquiring open-access papers, proposing crops, reviewing panel boundaries for completeness of elements like axes and labels, and filtering statistical visualization panels Each panel is scored by a local vision model prompt to assign metrics such as clarity/integrity and data purity as construction-time filters The resulting 6,740 reference packages establish coverage for the presentation-reference protocol, not recovered underlying datasets Evaluation Protocol The evaluation protocol employs two complementary settings: image-only and caption-assisted reconstruction The model receives either only the target panel in the image-only setting or the target panel along with optional metadata like captions, DOI, and journal information in the caption-assisted setting All tasks require executable, editable plotting logic, and submissions are invalid if they rely on direct image dependencies or raster tracing instead of meaningful plotting primitives The evaluation unit is an executable presentation program rather than a data estimate or copied image Key Findings The evaluation reveals that capable general-purpose VLMs can recover substantial visual structure from final scientific panels, even without raw data or author source code Specifically, Claude Opus 4.7 achieved the highest image-only Overall score (83.50), and Claude Opus 4.6 led Valid (95.25) and Type (67.17) scores in the image-only setting Caption assistance improved Overall for 8 of 14 models, with the largest gains seen for Gemma-4-31B-IT (+7.02) and Ovis2.6-80B-A3B (+5.50) Reasoning-guided prompting improved the end-to-end score for all four tested models, showing gains ranging from +1.78 to +6.
Title and authors: Tom: So, we're looking at this paper called SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code. Basically, it’s about how AI can take a finished scientific chart and turn it back into editable code that looks the same.
Jane: That sounds really interesting because most of the time, we just look at the picture and try to describe what's in it, but this paper is testing if an AI can actually build a working program that makes that exact figure again.
Lu: Exactly. The core idea here is moving beyond just looking at a picture and asking what it *shows*. They’re focusing on presentation recovery, which means making sure the arrangement of the data, the labels, everything—it all stays exactly as it was in the original panel.
Meng: I mean, that sounds like a lot of precise control for an AI to manage. What they’re claiming is they built this benchmark to test that specific kind of reconstruction ability rather than just general chart understanding.
Lalam: Right, and the paper sets up a whole system where specialized AI agents—Codex agents—are tasked with generating these programs. They aren't just guessing; they are being instructed to create a silver-standard presentation program that captures the geometry, the hierarchy, and all those visual elements without trying to recover the original measurements or author source code.
Tom: That sounds like a good way to stop the AI from just spitting out some random numbers and start forcing it into a structured output. They’re building this benchmark by converting six thousand seven hundred forty published panels into these auditable reference packages <ref:2609.08155#pg1>.
Jane: Sixty-seven hundred forty panels is a big number. What they’ve done there is create this balanced test set called SCIFIGUREBENCH, which has three hundred thirty seven panels covering thirty one chart subtypes and five different scientific domains across three complexity levels.
Lu: That diversity in the test set is important because it shows how robust the recovery process needs to be across different types of charts, not just one specific kind of visualization.
Meng: But I gotta ask, what does this reconstruction actually mean for deployment? If an AI can reliably generate code that renders a figure perfectly, does that make it more useful for researchers than just generating text descriptions?
Title and authors: Tom: Well, the paper is testing this across fourteen different zero-shot models using both image-only and caption-assisted settings. They found that even without any raw data or original source code, capable general purpose vision models can recover a substantial visual structure from those final published panels.
Jane: So they ran the evaluation in two ways—just the image, and then with some extra context like captions and journal information in the caption-assisted setting. It shows that adding that context helps a lot for some models.
Lu: They found that Claude Opus four point seven actually achieved the highest overall score of eighty-three point five zero in the image-only setting, and Claude Opus four point six led in both valid and type scores within that same image-only test <ref:2609.08155#pg3>.
Meng: That’s interesting data on which model is best at this reconstruction task, but what about the others? Are we talking about models that can actually produce runnable Python code, or just pretty pictures?
Lalam: The evaluation protocol is strict about execution. Submissions are invalid if they don't rely on meaningful plotting primitives and executable logic; it has to be an actual presentation program you can run in a working directory.
Tom: That execution gate is key. It’s not enough for the model to *think* it can make code; it has to actually produce code that runs and produces a PNG or PDF artifact. That moves the evaluation away from just estimating data and toward generating something tangible, like an editable presentation program.
Jane: And when we look at how caption assistance helps, they saw improvements for eight out of the fourteen models in that setting. The biggest gains were seen with Gemma four 31B IT and Ovis2 point 6 80B A3B models, which showed gains of seven point zero two and five point five zero respectively <ref:2609.08155#pg3>.
Lu: That suggests that having a good caption helps the model understand the scientific context needed for accurate reconstruction, especially when it comes to those specific reasoning-guided prompts they tested. They also found that using a bounded visual reconstruction plan in their prompting strategy improved end-to-end scores across all four models, with gains ranging from one point seven eight up to six point three seven for Gemma four 31B IT <ref:2609.08155#pg3>.
Meng: I’m still worried about the scale. How much compute does this take? Running these complex agent pipelines and testing fourteen different models against a benchmark like SCIFIGUREBENCH sounds expensive when you try to deploy this at scale in a production environment.
Title and authors: Lalam: The paper addresses the construction cost by using a local Qwen family vision model as a construction-time filter, scoring panels on metrics like clarity, integrity, and data purity—basically using that local model just to score the input quality before the main reconstruction agents even start.
Tom: That sounds like an efficient way to prune bad inputs early on. But what about when we move from this benchmark to real research papers? Can a researcher actually use this system on their own unpublished work?
Jane: The protocol is designed to turn final published panels into auditable reference packages, which implies it’s meant for inspecting and reusing existing scientific literature, not necessarily for generating new figures from scratch. It’s about provenance.
Lu: The paper sets up a very clear pipeline: acquire papers, propose crops, review boundaries for elements like axes and labels to make sure they are intact before the reconstruction starts. This is a lot of manual review integrated into the process to ensure quality control at every step.
Meng: So it’s not just one big AI call; it’s this entire sequence of filtering, proposing crops, reviewing boundaries, then the agent writing code, then rendering and auditing. That level of pipeline complexity sounds like a lot of engineering overhead to manage reliably.
Tom: The authors are pushing the idea that this reconstruction-and-audit protocol creates these reference packages which contain six thousand seven hundred forty reviewed panels and the SCIFIGUREBENCH test set for people to inspect later <ref:2609.08155#pg1,this reconstruction-and-audit protocol>. That’s the main deliverable for auditing figure provenance.
Jane: So, to wrap up what we’ve heard about SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code, this work shows that current general purpose vision models can do more than just summarize charts; they can actually rebuild a presentation program that preserves the visual layout and reading order of scientific panels.
Lu: The improvement they found is in the method itself—using role specialized Codex agents to generate silver standard programs that focus on geometry and typography without claiming to recover original measurements or author source code, which keeps the output clean for auditing.
Meng: For deployment, the paper suggests this reconstruction-and-audit protocol turns final published panels into auditable reference packages, which is a valuable resource for verifying where scientific claims come from.
Title and authors: Lalam: And it establishes SCIFIGUREBENCH as a balanced test set across thirty one chart subtypes and five domains to properly evaluate this recovery ability under different conditions.
Tom: So, the implication is that this tool helps researchers inspect figures, but they’ve also highlighted that the process requires careful construction-time filtering and execution gates to ensure the output is actually useful code.
Jane: It gives us a concrete benchmark for how good AI is at preserving visual structure across different scientific fields when we ask it to generate presentation code.
Lu: And looking forward, they are pointing toward improving the reconstruction of specific chart subtypes, like enhancing the ability to preserve spatial topology and compositional structure, because those were identified as major points of structural loss in high complexity panels.
Meng: So the future work is about fine-tuning these models specifically to handle those tricky visual structures that tend to get mangled when they are reconstructed into code.
Tom: That’s the direction it’s heading—moving from general structure recovery toward handling the specific, complex visual details that really matter in scientific figures.
Jane: So for anyone listening who just wants a concrete idea of what this paper is about, SciFigure2Code is a way to test if AI can accurately recreate the presentation logic of scientific charts into working code.
Lu: It’s about creating auditable reference packages from published panels, using specialized agents and strict execution rules to ensure the reconstructed program is editable and accurate.
Meng: The caveat they put out is that this benchmark focuses on presentation recovery, not recovering underlying datasets, which keeps the scope clear for what the AI can actually achieve.
Lalam: It shows that caption assistance helps, especially when combined with reasoning guided prompting, but it still requires a lot of careful engineering to manage the whole pipeline from acquisition to audit.
Tom: So if you’re interested in how we test these models—what data they use and what metrics they look at—this paper is the place to see how they set up that entire system.
Jane: It sounds like a solid piece of work for anyone who wants to understand the current state of multimodal AI applied specifically to scientific visualization tasks.
The paper's summary: Tom: So we're looking at this paper called SciFigure2Code, which is essentially creating a benchmark to see how well AI can rebuild scientific charts into actual, editable code.
Jane: That sounds like a lot of precise control for an AI to manage, because most of the time we just look at a picture and try to describe what's in it.
Lu: Exactly. They’re focusing on presentation recovery, which means making sure the arrangement of the data and all those labels stay exactly as they were in the original panel.
Meng: I mean, that sounds like a lot of precise control for an AI to manage, because most of the time we just look at a picture and try to describe what's in it.
Lalam: Right, and the paper sets up this whole system where specialized AI agents are tasked with generating these programs. They aren't just guessing; they are being instructed to create a silver-standard presentation program that captures geometry and hierarchy without trying to recover the original measurements or author source code.
Tom: That sounds like a good way to stop the AI from just spitting out some random numbers and start forcing it into a structured output. They’re building this benchmark by converting six thousand seven hundred forty published panels into these auditable reference packages.
Jane: Sixty-seven hundred forty panels is a big number. What they’ve done there is create this balanced test set called SCIFIGUREBENCH, which has three hundred thirty seven panels covering thirty one chart subtypes and five different scientific domains across three complexity levels.
Lu: That diversity in the test set is important because it shows how robust the recovery process needs to be across different types of charts, not just one specific kind of visualization.
Meng: But I gotta ask, what does this reconstruction actually mean for deployment? If an AI can reliably generate code that renders a figure perfectly, does that make it more useful for researchers than just generating text descriptions?
Tom: Well, the paper is testing this across fourteen different zero-shot models using both image-only and caption-assisted settings. They found that even without any raw data or original source code, capable general purpose vision models can recover a substantial visual structure from those final published panels.
The paper's summary: Jane: So they ran the evaluation in two ways—just the image, and then with some extra context like captions and journal information in the caption-assisted setting. It shows that adding that context helps a lot for some models.
Lu: They found that Claude Opus four point seven actually achieved the highest overall score of eight point three five in the image-only setting, and Claude Opus four point six led in both valid and type scores within that same image-only test.
Meng: That’s interesting data on which model is best at this reconstruction task, but what about the others? Are we talking about models that can actually produce runnable Python code, or just pretty pictures?
Lalam: The evaluation protocol is strict about execution. Submissions are invalid if they don't rely on meaningful plotting primitives and executable logic; it has to be an actual presentation program you can run in a working directory.
Tom: That execution gate is key. It’s not enough for the model to think it can make code; it has to actually produce code that runs and produces a PNG or PDF artifact. That moves the evaluation away from just estimating data and toward generating something tangible, like an editable presentation program.
Jane: And when we look at how caption assistance helps, they saw improvements for eight out of the fourteen models in that setting. The biggest gains were seen with Gemma four 31B IT and Ovis2 point six 80B A3B models, which showed gains of seven point zero two and five point five zero respectively <ref:2609.08155#pg3>.
Lu: That suggests that having a good caption helps the model understand the scientific context needed for accurate reconstruction, especially when it comes to those specific reasoning guided prompts they tested. They also found that using a bounded visual reconstruction plan in their prompting strategy improved end-to-end scores across all four models, with gains ranging from one point seven eight up to six point three seven for Gemma four 31B IT <ref:2609.08155#pg1>.
Meng: I’m still worried about the scale. How much compute does this take? Running these complex agent pipelines and testing fourteen different models against a benchmark like SCIFIGUREBENCH sounds expensive when you try to deploy this at scale in a production environment.
Lalam: The authors addressed the construction cost by using a local Qwen family vision model as a construction-time filter, scoring panels on metrics like clarity and integrity before the main reconstruction agents even start.
The paper's summary: Tom: That sounds like an efficient way to prune bad inputs early on, but what about when we move from this benchmark to real research papers? Can a researcher actually use this system on their own unpublished work?
Jane: The protocol is designed to turn final published panels into auditable reference packages, which implies it’s meant for inspecting and reusing existing scientific literature, not necessarily for generating new figures from scratch. It’s about provenance.
Lu: The authors are pushing the idea that this reconstruction-and-audit protocol creates these reference packages which contain six thousand seven hundred forty reviewed panels and the SCIFIGUREBENCH test set for people to inspect later—that’s the main deliverable for auditing figure provenance.
Meng: So it’s not just one big AI call; it’s this entire sequence of filtering, proposing crops, reviewing boundaries, then the agent writing code, then rendering and auditing. That level of pipeline complexity sounds like a lot of engineering overhead to manage reliably.
Lalam: It shows that caption assistance helps when combined with reasoning guided prompting and it still requires a lot of careful engineering to manage the whole pipeline from acquisition to audit.
Tom: So if you’re interested in how we test these models—what data they use and what metrics they look at—this paper is the place to see how they set up that entire system.
Jane: It sounds like a solid piece of work for anyone who wants to understand the current state of multimodal AI applied specifically to scientific visualization tasks.
Lu: And looking forward, they are pointing toward improving the reconstruction of specific chart subtypes, like enhancing the ability to preserve spatial topology and compositional structure because those were identified as major points of structural loss in high complexity panels.
Meng: So the future work is about fine-tuning these models specifically to handle those tricky visual structures that tend to get mangled when they are reconstructed into code.
Tom: That’s the direction it’s heading—moving from general structure recovery toward handling the specific, complex visual details that really matter in scientific figures.
Jane: So for anyone listening who just wants a concrete idea of what this paper is about, SciFigure2Code is a way to test if AI can accurately recreate the presentation logic of scientific charts into working code.
Lu: It’s about creating auditable reference packages from published panels, using specialized agents and strict execution rules to ensure the reconstructed program is editable and accurate.
The paper's improvements: Tom: We just talked about how they built this benchmark to test if AI can rebuild scientific charts into actual code, and now we're looking at how they suggest making that process better with some specific fixes.
Jane: So, what are these suggested improvements? Are they just tweaking the prompts or is there a bigger change in how the AI is supposed to learn this skill?
Lu: They’re suggesting a few key adjustments. First, they want those specialized Codex agents to be explicitly tasked with generating silver-standard presentation programs that focus on geometry and typography without claiming they recovered original measurements.
Meng: So, they're trying to make sure the output is an auditable reference package instead of just some potentially misleading data recovery attempt. That sounds like a necessary guardrail for us when we deploy anything.
Lalam: Right, and second, they want execution-gated results for all models. This means the generated programs have to use editable plotting primitives and run in an empty working directory to save both PNG and PDF artifacts.
Tom: That gate is crucial because it forces the evaluation to focus on an actual executable presentation program rather than just a data estimate or a copied image. It’s about making sure the output is tangible.
Jane: I see that, but they also suggest enhancing the two-stage plan-then-code prompting by explicitly requesting a bounded visual reconstruction plan in the first stage to improve performance across all four models.
Lu: That addresses those structural losses they found, and it actually helped with end-to-end scores for models like Gemma four 31B IT, showing gains up to six point three seven <ref:2609.08155#pg1>.
Meng: If that planning step helps performance, then maybe we should focus our engineering efforts there instead of just tuning the final code generation part.
Tom: Exactly. And they also integrated a local Qwen family vision model as a construction-time filter to score panels on things like clarity and data purity before the main reconstruction agents even start.
Jane: That sounds like using that local model just to score the input quality, which is a smart way to prune bad inputs early on. It makes sense for reducing wasted compute time down the line.
Lu: They also developed prompt templates that condition the reconstruction task based on modality; specifically augmenting image-only prompts with the target panel’s caption and content summary in caption-assisted settings.
Tom: That sounds like a practical way to help disambiguate scientific labels when we're working with complex images. It helps the AI know what context to use.
Meng: So, they're moving toward better context awareness instead of just hoping the model can guess the meaning from the image alone. That makes more sense for real-world data handling.
Jane: And finally, they fine-tuned models to recognize things that are scientifically meaningful only with paper context; specifically gene and protein symbols, physical units, treatment groups, and significance markers.
Lalam: That’s addressing that difficulty of handling labels that are meaningful only with paper context during reconstruction. I'm pretty sure my literature training helps me see those specific markers when they're in the right format.
Tom: So they’re focusing on making sure the AI understands the actual scientific language, not just generic visual patterns. That’s a solid improvement for accuracy.
Jane: It sounds like this work is moving toward building an AI that isn't just looking at shapes and lines, but truly understanding the meaning behind the scientific notation in those figures.
Lu: And it points toward improving how we handle specific chart subtypes, focusing on preserving spatial topology and compositional structure because those were identified as major points of structural loss in high complexity panels.
Tom: So to wrap up these improvements: they're adding strict planning steps, execution gates for running actual code, using local models for input scoring, and fine-tuning for scientific context recognition.
Jane: It gives us a much clearer path toward building an AI that can reliably reconstruct the presentation logic of scientific charts into working code.
Lu: And looking forward, they are pointing toward improving those specific structural losses in complex panels, which is where the real challenge lies for high-quality outputs.
Conclusion: Tom: So we’re wrapping up on SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code, which is basically testing if AI can rebuild scientific charts into actual, editable code.
Jane: It sounds like this paper is giving us a solid way to measure how well multimodal AI can handle the complex visual language of scientific figures.
Lu: They’ve shown that capable general purpose vision models can recover a substantial visual structure even without any raw data or original source code.
Meng: So, what does this mean for deployment? Can we actually trust an AI to generate production-ready code from a final figure?
Tom: The evaluation showed that Claude Opus four point seven achieved the highest overall score of eight point three five in the image-only setting, and Claude Opus four point six led in both valid and type scores there.
Jane: So it seems that for now, models like Claude are really strong at capturing the visual layout when they only see an image.
Lalam: I’m pretty sure that's because of the caption assistance they tested; it helped eight out of fourteen models improve their overall scores by adding context.
Lu: That context matters a lot for disambiguating scientific labels, which is where many of these models get tripped up when they only see a picture.
Meng: From an engineering standpoint, I still have to wonder about the cost of running all those complex agent pipelines and testing fourteen different models against a benchmark like SCIFIGUREBENCH.
Tom: It’s expensive, for sure, but the authors did address that by using a local Qwen family vision model as a construction-time filter to score panels before the main work starts.
Jane: So they’re trying to manage that scale issue by filtering bad inputs early on using a lighter model. That makes sense for reducing wasted compute time down the line.
Lu: And looking ahead, they are pointing toward improving those specific structural losses in complex panels, focusing on preserving spatial topology and compositional structure because those were identified as major points of structural loss.
Meng: So the future work is about fine-tuning these models specifically to handle those tricky visual structures that tend to get mangled when they are reconstructed into code.
Tom: That’s the direction it’s heading—moving from general structure recovery toward handling the specific, complex visual details that really matter in scientific figures.
Jane: So for anyone listening who just wants a concrete idea of what this paper is about, SciFigure2Code is a way to test if AI can accurately recreate the presentation logic of scientific charts into working code.
Lu: It’s about creating auditable reference packages from published panels, using specialized agents and strict execution rules to ensure the reconstructed program is editable and accurate.
Lalam: I'm pretty sure that this benchmark gives us a much clearer yardstick for measuring AI performance in this specific area of scientific visualization recovery.
More episodes
- 2606.27866-FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- 2607.04680-A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- 2607.04919-When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters
- 2607.05025-Uncertainty-aware damage identification in short-span bridges via physics-informed variational autoencoder
- 2607.12304-What Does a Temporal Benchmark Score Measure? Decomposing Channel Use in Video VLM Evaluation
- 2608.02877-Graph-of-Thoughts for LLM Causal Discovery with a Fairness-Pathway Case Study
- 2110.08294-Coherence boosting: When your pretrained language model is not paying enough attention
- 2210.01293-ThinkSum: Probabilistic reasoning over sets using large language models
- 2504.15270-Quicksviewer: An LMM with Early Video Compression using Annealed Gumbel-Softmax
- 2508.10683-Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages