PubMedCausal: A Span-Level Annotated Corpus for Causal Relation Extraction in Biomedical Text

summary

The gist

Causal relation extraction (CRE) is central to biomedical text mining, but current resources often conflate causal relations with broader associations…

In short

PubMedCausal is a new, span-level annotated corpus of 30,000 rows from PubMed abstracts designed for causal relation extraction in biomedical text. It annotates full cause and effect spans, causality type, and sententiality. The study shows that biomedical encoders perform best for detection (PubMedBERT) and generative models are strong for extraction (DeepSeek-R1-32B). Results indicate that implicit and inter-sentential causality remains challenging.

Key concepts

Causal Relation Extraction (CRE)
This is the task of identifying cause-and-effect relationships within text. In biomedical research, this is crucial for understanding how diseases develop or how drugs work. Current resources are often too broad or sentence-level, which limits their usefulness for accurately finding these specific causal claims in complex scientific writing.
PubMedCausal Corpus
This is a newly created dataset of 30,000 paragraph-level rows from PubMed abstracts. Each row is annotated with whether it contains a causal relation and, if so, the exact text spans for the cause and effect. It also classifies the relationship by type (explicit/implicit) and structure (intra/inter-sentential), making it detailed enough for advanced evaluation.
Span-Level Annotation
Instead of just labeling an entire sentence as causal, this method annotates the specific words that constitute the cause and effect. This allows researchers to evaluate models not only on whether they can detect a claim but also on their ability to precisely extract the exact phrases representing that cause and effect.
Causality Types
The paper distinguishes between different ways causality is expressed in text: explicit versus implicit, and intra-sentential versus inter-sentential. Explicit means the cause and effect are stated clearly in the same sentence. Implicit means the relationship must be inferred by the reader. Intra-sentential relates to two parts of a single sentence, while inter-sentential refers to relationships between different sentences.

Terminology used across episodes

This episode discusses

The paper

PubMedCausal: A Span-Level Annotated Corpus for Causal Relation Extraction in Biomedical Text · Read on arXiv

Ifeoluwa Kunle-John, Josiah Paul, Oluwatosin Agbaakin, Peter Aina, Ikenna Odezuligbo, Sydney Anuyah

Edyah Limited University of Ibadan Indiana University Creighton University

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "PubMedCausal: A Span-Level Annotated Corpus for Causal Relation Extraction in Biomedical Text".

Jane: Here's the gist of the paper. Causal relation extraction (CRE) is central to biomedical text mining but current resources often conflate causal relations with broader associations, restrict annotation to sentence-level examples, or focus mainly on explicit causal cues Causal relation extraction (CRE) is central to biomedical text mining but current resources often conflate causal relations with broader associations, restrict annotation to sentence-level examples, or focus mainly on explicit causal cues Identifying causality in text requires attention to lexical choice, syntactic composition, discourse relations, and modality since each can substantially alter the meaning, direction, and strength of a causal claim Reliable CE in biomedical contexts can support downstream applications such as drug discovery, disease management, clinical decision-making, and precision medicine The abundance of biomedical data presents a growing scale problem for manual annotation Current automated systems remain limited in their ability to capture this complexity We did not design a new model to resolve this challenge instead we provide a resource for evaluating CE enabling deeper examination of causality and uncertainty types while offering a baseline for future work Corpus Construction and Annotation Protocol PubMedCausal is introduced as a span-level annotated corpus built from PubMed abstracts containing 30,000 paragraph-level rows including 3,945 causal rows and 6,491 adjudicated cause–effect pairs Each causal relation is annotated with full-text cause and effect spans causality type and sententiality enabling evaluation of both causal detection and full-span causal extraction The corpus was constructed from PubMed abstracts retrieved using the keyword “causality” from January 1st to September 3rd, 2025 which returned 24,603 free full-text abstracts Rows containing no causal relation were retained in the corpus since the corpus is intended to support both CE and causal/non-causal classification Annotators were instructed to read each row as a single local discourse unit and extract a tuple with four fields Cause Effect Type Sententiality Causal relations were classified as either explicit or implicit and intra-sentential or inter-sentential based on the markers present in the text Experimental Evaluation Results The results show that biomedical encoders are strongest for causal detection with PubMedBERT reaching an F1 score of 0.7391 For span-level extraction the best generative baseline is DeepSeek-R1-32B with few-shot prompting reaching a Cosine Pair F1 of 0.6765 Causal detection is difficult under class imbalance long causal spans implicit causality inter-sentential relations and prompt sensitivity The model performed better on direct and structurally simpler cases while inter-sentential causality remains particularly challenging with a baseline F1 score of only 5% Explicit causality achieved an F1 score of 88.03% while implicit causality achieved an F1 score of 39.20% Limitations and Future Directions PubMedCausal is limited to English PubMed abstracts retrieved with the keyword “causality” which means the corpus is not a representative sample of all biomedical writing The sampling strategy introduces a scope limitation as it overrepresents papers concerned with causal inference and underrepresents ordinary biomedical causal statements Preprocessing choices such as removing rows containing numbers or non-ASCII characters may exclude quantitative biomedical causal statements involving measurements or trial outcomes The class imbalance is significant as explicit intra-sentential causal relations are much more common than implicit and inter-sentential relations making the latter harder to evaluate reliably Future extensions could reduce sampling limitations by adding randomly sampled PubMed abstracts and retrieval queries based on broader causal trigger terms such as “induces” or “leads to” The study concludes that there is still a need to develop frameworks that can handle automated extraction at a higher level of precision and recall Dataset Release PubMedCausal will be released via Hugging Face Datasets at

repository link to be added upon acceptance: including the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID the row text a binary causal label and for causal rows a list of cause–effect pair tuples each containing cause span effect span expression type and sententiality Pre-constructed train and test split indices are provided alongside the data Raw abstract text is released where permitted under PubMed’s open access policy for remaining records PubMed IDs are provided to allow retrieval The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID the row text a binary causal label and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID, the row text, a binary causal label, and—for causal rows—a list of cause–effect pair tuples, each containing cause span effect span expression type (Explicit/Implicit), and sententiality (Intra/Inter) The release includes the full 30,000-row corpus in JSON Lines format with fields for the source PubMed abstract ID,

Paper summary: Tom: So, to recap this PubMedCausal paper, they are essentially saying that existing resources are too narrow. They introduce this corpus to fix that by annotating relations at the span level in biomedical text.

Jane: It's not just about finding any link; it’s specifically about distinguishing between true causal claims and broader associations that might look similar on the surface. The authors claim this new resource is a better tool for testing how well models can actually recover causality as it appears in real biomedical writing.

Lu: They point out that previous datasets like BC5CDR or BioRED focus more on chemical-gene relationships or entity-level associations, which isn't enough when you need to evaluate causal claims expressed naturally in complex paragraphs.

Meng: The specific construction method they used is key here. They retrieved abstracts from PubMed using the keyword “causality” between January 1st and September 3rd, two thousand twenty-five. That gave them these thirty thousand paragraph-level rows.

Lalam: And they didn't just label the whole row as causal or not; they instructed their annotators to look at each row as a single local discourse unit and extract four fields: cause, effect, type of causality, and sententiality.

Tom: That means they are classifying relations as either explicit or implicit, and whether those causes and effects are in the same sentence or across sentences. This level of detail is what makes it span-level annotation.

Jane: What matters for us is that this allows researchers to test detection and full-span extraction simultaneously, which is a much more rigorous way to see if an AI can truly capture the nuance of biomedical causality.

Conclusion: Tom: So, looking at the whole PubMedCausal paper by Kunle-John et al., what we’re seeing is that this resource moves beyond just finding simple links and focuses on the structural complexity of causal claims in medical literature.

Jane: The authors argue that this span-level annotation is necessary because current tools often miss implicit causality or confuse it with other types of associations. It provides a benchmark for evaluating models against these harder, more realistic textual challenges.

Lu: The implication here is that if a model can perform well on this kind of dataset, it means it's learning something deeper about the structure of scientific writing than just surface-level pattern matching.

Meng: From an engineering standpoint, the authors show that while biomedical encoders like PubMedBERT do well for detection—reaching an F1 score of zero point seven three nine one—the span-level extraction is harder, with a generative model like DeepSeek-R1-32B getting a Cosine Pair F1 of zero point six seven six five.

Lalam: They also tested transfer learning by seeing how well encoders trained on PubMedCausal perform on external causal relation datasets, which shows the resource has some generalizability for cross-dataset evaluation.

Tom: Overall, this paper is important because it gives us a dedicated resource to properly test if AI can actually read and understand the way causality is expressed in biomedical text, moving past simpler association tasks.

More episodes

← Home