Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning
Sahil Al Farib, Momota Ahsana Meem, Sheikh Redwanul Islam, Md. Tanvir Raihan
cs.AI, cs.CL
Submitted: 2026-08-04
License: http://creativecommons.org/licenses/by/4.0/
The gist: Lecture videos distribute knowledge across speech, slide text, diagrams, equations, and presentation order, which transcript-only retrieval does not fully preserve.
Terminology
Abstract
Lecture videos distribute knowledge across speech, slide text, diagrams, equations, and presentation order, which transcript-only retrieval does not fully preserve. This paper presents an evidence-grounded multimodal pipeline that transcribes lectures, selects semantic anchors, applies optical character recognition (OCR), and uses a vision-language model to extract only concepts and typed relationships supported by transcript, OCR, or visual evidence. Mentions are validated and canonicalized into a provenance-rich knowledge graph. On three neural-network lectures, the pipeline processed 3,118 frames, 756 transcript segments, and 559 anchors. It retained 1,022 concept and 312 relationship mentions, yielding 172 canonical concepts and 282 relationships with 90.38% endpoint coverage. A preliminary three question retrieval test achieved 100% top-1 and top-3 accuracy and 100% mean top-5 recall. The contribution is an auditable construction method rather than a state-of-the-art performance claim.
Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- Robust Speech Recognition via Large-Scale Weak Supervision
- Qwen2.5-VL Technical Report
- Multi-Modal Knowledge Graph Construction and Application: A Survey
- LLM-empowered knowledge graph construction: A survey
- An Optimized Pipeline for Automatic Educational Knowledge Graph Construction
- A Survey of Multimodal Retrieval-Augmented Generation
- Retrieval-Augmented Generation for Natural Language Processing: A Survey
- Kuaipedia: a Large-scale Multi-modal Short-video Encyclopedia
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems