Subgraph-Aware Training of Language Models for Knowledge Graph Completion Using Structure-Aware Contrastive Learning

arXiv:2407.12703 · cs.CL, cs.AI · Submitted 2024-07-17 · Read on arXiv

cs.CL, cs.AI

Submitted: 2024-07-17

Updated: 2025-01-31

Code: https://github.com/meaningful96/SATKGC

Terminology

• Knowledge Graph — 5× in this paper · explained in 7 episodes
• Contrastive Learning — 2× in this paper · explained in 6 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Knowledge Graph Completion (KGC) — 1× in this paper · explained in 2 episodes
• Knowledge graphs — 1× in this paper · explained in 2 episodes

Sources

• Structure Aware Negative Sampling in Knowledge Graphs
• A Simple Framework for Contrastive Learning of Visual Representations
• CERT: Contrastive Self-supervised Learning for Language Understanding
• SimCSE: Simple Contrastive Learning of Sentence Embeddings
• Self-Supervised Contrastive Representation Learning for 3D Mesh Segmentation
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• LoRA: Low-Rank Adaptation of Large Language Models
• Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective
• Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax
• Representation Learning with Contrastive Predictive Coding
• Balanced Meta-Softmax for Long-Tailed Visual Recognition
• MPNet: Masked and Permuted Pre-training for Language Understanding
• Equalization Loss for Long-Tailed Object Recognition
• Language Models as Knowledge Embeddings
• Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax
• Understanding Negative Sampling in Graph Representation Learning
• KG-BERT: BERT for Knowledge Graph Completion
• HaSa: Hardness and Structure-Aware Contrastive Knowledge Graph Embedding
• Contrastive Learning of Sentence Embeddings from Scratch
• Making Large Language Models Perform Better in Knowledge Graph Completion

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering