Plain Transformers are Surprisingly Powerful Link Predictors
The PENCIL model uses a standard Transformer architecture to predict links by analyzing small, local subgraphs around candidate nodes. It proves that plain Transformers can effectively perform complex link prediction tasks without needing complicated structural encodings or large memory usage. This shows simple designs can achieve high predictive power under strict deployment limits.