Are LLM Assistants Regulatory Ready? Answers from regenold's 2026 EU AI Act Q&A Challenge
cs.AI, cs.MA
Submitted: 2026-11-03
Updated: 2026-11-03
Code: https://github.com/regenold-GmbH/eu-ai-act-benchmark-2026
Terminology
Sources
- LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
- LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain
- A Reasoning-Focused Legal Retrieval Benchmark
- RIRAG: Regulatory Information Retrieval and Answer Generation
- FDARxBench: Benchmarking Regulatory and Clinical Reasoning on FDA Generic Drug Assessment
- AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliance
- AI Act Evaluation Benchmark: An Open, Transparent, and Reproducible Evaluation Dataset for NLP and RAG Systems
- Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
- Enabling Large Language Models to Generate Text with Citations
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
- Holistic Evaluation of Language Models
- MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs
- Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
- Measuring Sycophancy of Language Models in Multi-turn Dialogues
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents