Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank
cs.CL, cs.LG
Submitted: 2024-10-30
Updated: 2026-11-18
Code: https://github.com/gkamradt/LLMTest_
Terminology
Sources
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- Neural Machine Translation by Jointly Learning to Align and Translate
- Explainable ICD Coding via Entity Linking
- Asymmetric Loss For Multi-Label Classification
- Automated clinical coding using off-the-shelf large language models
- A Survey on Mixture of Experts in Large Language Models
- Revisiting Transformer-based Models for Long Document Classification
- Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- The Llama 3 Herd of Models
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- PLM-ICD: Automatic ICD Coding with Pretrained Language Models
- Mixtral of Experts
- MEGClass: Extremely Weakly Supervised Text Classification via Mutually-Enhancing Text Granularities
- Decoupled Weight Decay Regularization
- CoRelation: Boosting Automatic ICD Coding Through Contextualized Code Relation Learning
- ICDBigBird: A Contextual Embedding Model for ICD Code Classification
- CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes
- Modelling Temporal Document Sequences for Clinical ICD Coding
- Mimic-IV-ICD: A new benchmark for eXtreme MultiLabel Classification
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data