Integrating Implicit and Explicit Relational Biases through Graph-Based Multiple Instance Learning: A Case Study in Skin Lesion Diagnosis

The paper Integrating Implicit and Explicit Relational Biases through Graph-Based Multiple Instance Learning: A Case Study in Skin Lesion Diagnosis by Rafał Buler, Jakub Buler, Michał Grochowski, and Maciej Bobowicz (Gdańsk University of Technology and Medical University of Gdańsk) investigates a dual-level relational framework for image classification that bridges implicit representation learning and explicit structural modelling.

The study addresses a key open question: it remains unclear whether relational modelling at the feature extraction stage alone is sufficient for downstream tasks, or whether combining it with explicit graph-based aggregation provides additional benefits. The authors position their work within three complementary paradigms for medical image analysis: (1) a conventional pipeline based on pretrained fully convolutional feature extraction, (2) a self-supervised approach learning patch-level representations through masked reconstruction that implicitly capture local spatial context without explicit relational modelling, and (3) a two-stage strategy that augments learned embeddings with graph-based aggregation to introduce explicit relational inductive bias between patches prior to classification.

The study uses two public benchmarks: ISIC-2018 Challenge Task 3 dataset and the ISIC-2019 dataset. ISIC-2018 consists of 10,015 training and 1,514 test images across seven classes, while ISIC-2019 comprises 25,331 training and 8,238 test images, extending the taxonomy with one additional class and a further 'unknown' category. The unknown samples were excluded because they lacked training data. For self-supervised pretraining, images from both datasets were combined and used jointly to train the masked autoencoder, with duplicate images removed. Since this stage is label-free, no annotation information was used and therefore no label leakage occurs. All images were centre-cropped into a square, resized to 224 × 224 px, and normalised using ImageNet statistics, with class imbalance addressed via weighted sampling.

Patch-level representations were learned using a convolutional masked autoencoder with a ViT backbone (ConvMAE). The authors explored masking ratios (0.25–0.75) and reconstruction losses, including mean squared error and its normalized variant, with model selection based on linear probing performance on ISIC-2018. A masking ratio of 0.50 yielded the lowest reconstruction loss and the resulting encoder was frozen and used across all experiments. Each image was decomposed into 14 × 14 = 196 non-overlapping patches (size 16 × 16 each), producing embeddings xi of dimensionality d = 768, which naturally supports a MIL formulation.

The attention-based MIL (AMIL) model treats each image as a bag of patches with instances treated as independent. The attention weight for each instance is computed as:

ai = exp(w⊤ tanh(Vxi)) / Σj exp(w⊤ tanh(Vxj))

with the image-level representation obtained by pooling the patch embeddings using an attention mechanism: z = Σi ai xi. This formulation is permutation-invariant with respect to patch ordering and treats all patch embeddings as independent instances, without explicitly modelling spatial or semantic relationships, serving as a reference point for assessing explicit relational modelling.

Three graph construction strategies were evaluated: (1) random graphs where each node vi is connected to up to r randomly sampled nodes (r ∈ 1, 2, 3, 4), serving as a non-structured baseline to isolate connectivity effects; (2) grid graphs defined using the original 14 × 14 spatial layout with 4-neighbourhood or 8-neighbourhood connectivity; (3) k-nearest neighbour graphs defined in feature space based on Euclidean distance where edges connect each node to its k nearest neighbours.

Graph-structured patch representations were processed using graph convolutional networks (GCN), graph attention networks (GAT), and graph transformer-based architectures with message passing defined as:

hi(l+1) = ϕ(hi(l), hj(l) (vi, vj) ∈ E)

After L layers, node embeddings were aggregated using the same attention-based MIL pooling as in the baseline to ensure differences arise solely from relational modelling via message passing. To isolate the effect of relational learning from increased capacity, a setting with frozen GNN layers was also evaluated.

A unified protocol was applied across all models, with tuning performed using five-fold CV on the training sets, with balanced accuracy as the selection metric.

On ISIC-2018, the conventional EfficientNet-B3 baseline achieved a balanced accuracy of 76.17%, which the authors note outperforms prior deep hierarchical approaches (72.60%) but trails ensemble methods (78.50–86.20%). The base AMIL model achieved a mean balanced accuracy (BAcc) of 77.12%, outperforming a conventional end-to-end CNN baseline. Introducing explicit relational modelling further improved performance: random graphs yielded moderate gains (best at r = 2) with 78.24%, while the highest BAcc of 79.27% was obtained with a grid graph without diagonal connections combined with GAT layers. The frozen GNN configuration caused a considerable decrease to 34.28%, demonstrating that learned relational modelling, rather than augmented capacity, is a primary factor contributing to performance. The authors state that grid-based MIL outperforms prior single-model approaches and remains competitive with ensemble methods that aggregate predictions from up to 90 deep neural models.

On ISIC-2019, the base AMIL achieved 59.84%, while the proposed approach achieves strong performance (60.67% BAcc) with a kNN graph (k = 4) processed with GAT layers, surpassing several existing methods and remaining competitive with more complex ensemble-based solutions (the top ensemble leaderboard entries range from 56.90% to 63.60%).

The authors conclude that self-supervised patch representations learnt via masked image modelling provide a robust foundation for downstream classification and that incorporating explicit relational modelling through GNNs further improves performance. The substantial drop from freezing GNN parameters underscores the importance of learnt message passing for capturing meaningful inter-patch relationships. The findings indicate that relational information can be effectively encoded in an intermediate manner during the SSL and enhanced through additional explicit graph modelling. The framework "achieves competitive results compared to previous single-model deep learning approaches and ensemble methods, highlighting the value of combining self-supervised learning with graph-based relational modelling in image analysis. The study emphasises the complementary roles of implicit and explicit relational modelling in patch-based classification and exhibits strong generalisation capabilities, demonstrating that it can be applied to model data relations and solve diagnostic and classification tasks across other medical domains."

The paper was accepted for presentation at the 21st International Conference on Computational Intelligence Methods for Bioinformatics and Biostatistics (CIBB 2026), and the authors declare no competing interests.

Improvements to AI systems:

• Integrate implicit relational learning (self-supervised masked autoencoding, e.g., ConvMAE) with explicit relational modeling (graph neural networks over image patches) rather than relying on either alone.

• Replace monolithic image encoders with a multiple-instance learning (MIL) framework: decompose images into 16×16 patches, generate 768-d embeddings via a frozen SSL encoder, and use attention-based MIL pooling to produce image-level representations.

• Build graphs over patch embeddings using multiple strategies—spatial grid graphs (4- or 8-neighbourhood) and feature-space k-nearest-neighbour graphs (k=4)—to capture both geometric and semantic inter-patch relations.

• Use graph attention networks (GAT) for message passing, with attention-weighted aggregation over node neighbours, as they empirically outperform GCN and graph transformer variants.

• Train the GNN layers end-to-end (not frozen); the sharp performance drop when freezing GNN weights (from 79% to 34% balanced accuracy) shows that learned relational weights are critical.

• Pretrain the patch encoder on large unlabeled datasets from multiple domains (e.g., combining ISIC-2018 and ISIC-2019) with masked reconstruction, using a masking ratio of 0.50, to capture transferable local spatial context without label leakage.

• Apply a unified hyperparameter selection protocol: five-fold cross-validation with balanced accuracy as the target metric, which handles class imbalance and avoids overfitting.

What the improved AI system can do:

• Achieve higher classification accuracy on medical image tasks (e.g., skin lesion diagnosis) than conventional CNNs and prior MIL baselines—reaching 79.27% balanced accuracy on ISIC-2018 and 60.67% on ISIC-2019 with a single model, competitive with much larger ensemble methods.

• Learn robust patch representations from unlabeled data, enabling effective use of scarce labeled medical datasets.

• Explicitly model both spatial layout (through grid graph edges) and semantic similarity (through kNN edges) between image regions, leading to better understanding of lesion structure and context.

• Generalize to other medical domains where data can be represented as a collection of interacting instances (e.g., histopathology tiles, radiology image regions), providing a plug-and-play framework for diagnostic classification and relation modelling.

Relational inductive biases are essential for capturing structural dependencies among data. This study investigates a dual-level relational framework for image classification, bridging the gap between implicit representation learning and explicit structural modelling. We begin by establishing a baseline using an EfficientNetB3 architecture. To move beyond standard convolutional biases, we adopt a patch-based strategy, employing a convolutional masked autoencoder to learn implicit inter-patch relationships through self-supervised reconstruction. We then extend this approach by incorporating explicit relational modelling, organizing the learned embeddings into various graph topologies, including grid-based, random, and k-nearest neighbour structures. Experimental results on the ISIC-2018 and ISIC-2019 skin lesion diagnosis benchmarks show that combining implicit inter-patch modelling with explicit graph-based message passing yields the best performance. On the ISIC-2018 test set, the baseline model achieves a balanced accuracy of 76.17%, which improves to 77.12% with implicit patch-based relational modelling. The fully integrated grid-structured Graph Attention Network further increases performance to 79.27%. Similarly, on ISIC-2019, the implicit approach reaches 59.84% balanced accuracy, while the combination of implicit and explicit modelling yields 60.67%.