Knowledge Distillation from Neural Networks to Differentiable Logic Networks
cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Terminology
Sources
- Adam: A Method for Stochastic Optimization
- Distilling the Knowledge in a Neural Network
- Model compression via distillation and quantization
- Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
- Distilling a Neural Network Into a Soft Decision Tree
- Making Logic Learnable With Neural Networks
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Learning Interpretable Differentiable Logic Networks for Time-Series Classification
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- FitNets: Hints for Thin Deep Nets
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning