Beyond Accuracy: Measuring Logical Compliance of Predictive Models
The Rule Violation Score (RVS) is a new metric that measures how well predictive models follow predefined logical or domain-specific rules, which is crucial for high-stakes applications. RVS treats hard and soft rules differently and can be computed using SQL. Experiments show that two models with similar accuracy can have very different levels of logical compliance, revealing important behavioral differences.