Safety Must Survive Self-Improvement: Why Failures Persist and How Agents Recover
This research tested recursive self-improvement (RSI) in stateful authorization tasks to see if safety could be maintained across generations. It found that failures persist due to historical scores favoring unsafe code and mechanisms like 'keep-after-rejection.' Recovery depends critically on deciding which program executes and which code supplies the next edit.