G-Loss: Graph-Guided Fine-Tuning of Language Models

arXiv:2604.25853 · cs.CL, cs.AI, cs.LG · Submitted 2026-04-28 · Read on arXiv

cs.CL, cs.AI, cs.LG

Submitted: 2026-04-28

Updated: 2026-08-27

Comments: 20 pages, Learning on Graphs Conference (LoG 2025)

Code: https://github.com/saditya13/G-Loss-LoG

License: http://creativecommons.org/licenses/by/4.0/

The gist: Traditional loss functions, including cross-entropy, contrastive, triplet, and su pervised contrastive losses, used for fine-tuning pre-trained language models such as BERT, operate only within local

Terminology

Abstract

Traditional loss functions, including cross-entropy, contrastive, triplet, and su pervised contrastive losses, used for fine-tuning pre-trained language models such as BERT, operate only within local neighborhoods and fail to account for the global semantic structure. We present G-Loss, a graph-guided loss function that incorporates semi-supervised label propagation to use structural relationships within the embedding manifold. G-Loss builds a document-similarity graph that captures global semantic relationships, thereby guiding the model to learn more discriminative and robust embeddings. We evaluate G-Loss on five benchmark datasets covering key downstream classification tasks: MR (sentiment analysis), R8 and R52 (topic categorization), Ohsumed (medical document classification), and 20NG (news categorization). In the majority of experimental setups, G-Loss converges faster and produces semantically coherent embedding spaces, resulting in higher classification accuracy than models fine-tuned with traditional loss functions.

Sources

Related papers