CourseGraph: Finding overlaps and differences in Computer Science courses across universities

The paper addresses a practical challenge in higher education student mobility. The authors note that "Student mobility programs such as Erasmus+ enable students to take courses at other universities, broadening their academic and cultural horizons. However, this flexibility also leads to a practical challenge: ensuring that students do not take courses elsewhere that substantially overlap with courses in their home curriculum. The authors explain that the evaluation of whether external courses overlap with home curriculum courses is typically performed manually by academic advisors or program coordinators, who determine whether the overlap between courses is small enough for all courses to be counted separately towards the student's degree requirements. In practice, this involves examining the intended learning outcomes (LOs), learning activities, and available educational material for the courses contained in the student's curriculum. The authors emphasize that this process is time-consuming, difficult to scale, and prone to subjectivity."

The authors also note a broader trend motivating the work: With the advent of Artificial Intelligence (AI) in the classroom, personalized and flexible learning pathways have become increasingly attainable, and students are encouraged to take greater ownership of their curricula by selecting elective courses that allow them to specialize in a field or diversify their knowledge across disciplines within their institution. This flexibility can take many forms, including "a specialization track or double degree that draws courses from a different department at their own university, or they may participate in student mobility programs such as Erasmus+ and take courses at partner universities abroad."

The paper presents CourseGraph, described as a methodology that automates the evaluation of external courses based on insights obtained from the process followed by curriculum administrators when assessing courses for inclusion in a degree program. The methodology processes course webpages as follows: "CourseGraph extracts information such as course titles, descriptions, and learning outcomes from the course webpage. Then, this information is represented semantically using a BERT-based language model, after which the pair-wise similarity between courses can be computed. This information is then used by a Random Forest classifier to determine whether a candidate course abroad overlaps with a course already contained in the student's curriculum."

The authors further explain: Beyond providing a way to identify overlap, CourseGraph also offers interpretability: it shows how different course components like LOs and the course description contribute to the overall overlap between two courses.

The paper addresses three research questions:

• RQ1) What criteria do curriculum assessors use to determine whether a course offered at another institution can be credited toward a given program?

• RQ2) How can the overlap in educational content between two university courses be determined using NLP, based on their textual descriptions, learning outcomes, and examination formats?

• RQ3) How well do overlaps between courses as indicated by NLP methods align with human judgments, when it comes to a) intra-university specialization tracks and b) international mobility programs?

The authors note that in the domain of recommendation systems in higher education, there has not been a prior effort in automating students' search of external courses, though some RS approaches use techniques such as Naive Bayes and K-means clustering. They note that advances in Natural Language Processing (NLP), particularly transformer-based models such as BERT, have greatly improved the ability to process large text volumes and that these models have become the new gold standard for RS.

For RQ1, the authors constructed a decision tree based on correspondence with the former program director who annotated their validation dataset. The decision-making process for assessing external courses at Lund University's C (Information and Communication Engineering) program involves: first determining whether a course is within the scope of the C program, or counted as external, where Each student is allowed to take external courses contributing to 15 European Credit Transfer and Accumulation System (ECTS) in total. If the course is part of the program, the program director identifies a corresponding course offered at LU, records the match, and transfers the level and credits. The level is defined as basic (G1), intermediate (G2) and advanced (A). Additionally, if the course is advanced and not part of the mandatory program, it is possible that it can be counted as a specialization track course if it is similar in content to a course in one of five tracks: Communication Systems (KS), Software Systems (PVS), Software (PVT), Security (SEC), and Usability, design and visualization (ADV).

The authors detail the overlap thresholding rules used by the program director: overlap below 20% (less than 1.5 ECTS) is disregarded; overlap between 20% and 66.7% warrants 2–5 ECTS; and overlap exceeding 66.7% (more than 5 ECTS) is treated as full equivalence. The paper states that in the rest of this paper, we focus mainly on identifying a content overlap, to address the first and third decision in Figure 1, and contends that automatically assigning course level would be relatively straightforward and require no additional neural network training since "course level can be reliably determined from existing data on the course page, through explicit mentions of the level (e.g., BSc/MSc), the presence of prerequisites (indicating higher difficulty), or implicit clues in the course title (e.g., 'Advanced' or numbered sequences like I/II)."

The CourseGraph methodology comprises four steps: 1) Data preprocessing, 2) Extracting Course Embeddings, 3) Computing the similarity between courses, 4) Training a classifier to identifying substantially overlapping courses.

Two datasets are used:

1. TU/e CS: The 180 ECTS Computer Science BSc program offered at Eindhoven University of Technology, where students follow a 125 ECTS domain-specific core program and have 45 ECTS elective credits. The university provides an overlap matrix, approved by the examination committee, to prevent students from taking courses with significant overlap in educational content. After processing, this resulted in 60 unique overlapping course pairs.

2. LU Erasmus+: the Information and Communication Engineering (C) program at Lund University, a 5-year program amounting to 300 European Credit Transfer System (ECTS) credits and the equivalent of a MSc degree upon completion. The dataset consists of approximately 25 exchange courses taken by students from the C program between 2018 and 2020, which were approved or partially approved by the former program director.

All course webpages were scraped and converted to JSON. To standardize course information across universities, DeepSeek-V3 was used to extract the course title, description, prerequisites, learning outcomes (LOs), and level, if these data fields were available on the webpage. The TU/e dataset trains the Random Forest classifier, while The LU Erasmus+ dataset is too small to support meaningful computation of metrics, so we use this as a hold-out test dataset that we analyze qualitatively.

The authors use semantic embeddings extracted by sBERT as opposed to word frequency based methods such as Bag-of-Words and TF-IDF features, since Semantic embeddings capture similarity between conceptually related expressions even when different terminology is used. Rather than representing a course as a single document-level embedding, "CourseGraph embeds each course component separately to improve the explainability of the decision-making process: this allows us to directly compare the similarity between, for example, the LOs of two courses, providing insight into which aspects of the two courses contribute most strongly to an overlap prediction."

Specifically: "The course title, prerequisites, and level are each represented by a single 768-dimensional embedding obtained by averaging the sentence embeddings within the corresponding text field. In contrast, learning outcomes are embedded individually, yielding an N × 768 representation, where N denotes the number of learning outcomes associated with the course. By preserving each learning outcome as a separate embedding, CourseGraph can identify which specific LOs are shared between two courses. Similarly, the course description is split into sentences, which are embedded separately."

Semantic similarity is computed using cosine similarity: sim(⃗a, ⃗b) = (⃗a ∗ ⃗b) / (⃗a ∗ ⃗b) — the authors adopt this metric because Sentence-BERT embeddings are explicitly optimized for cosine similarity comparison.

For LOs and course descriptions, which are represented as sets of embeddings, the similarity in LOs and course descriptions is expressed in a similarity matrix. Since the matrix size depends on the number of embeddings, the authors extract features from the matrix that do not depend on its size, namely "the maximum, minimum and mean similarity score, as well as the fraction of similarity scores above 0.5, 0.6, and 0.7. These six features are combined with the similarity scores obtained from the course title, level, and prerequisites, resulting in a 15-dimensional feature vector that serves as input to the classifier."

The classifier "is trained using labeled pairs of courses, where positive examples correspond to courses that have been identified as overlapping by curriculum administrators and negative examples correspond to courses without sufficient overlap. There are 60 positive pairs in the TU/e CS dataset, which are augmented with 180 negative pairs for training. Each pair of courses is represented by the 15-dimensional similarity feature vector described in Section III-C, and leads to a binary decision indicating whether the course pair is considered to be too similar in content for a student to take both: 'overlap' or 'no overlap'. Subsequent decisions about whether the matched course is mandatory or part of a specialization track are obtained directly from the curriculum metadata."

The authors evaluated several lightweight classification algorithms, including Logistic Regression, Random Forest, and XGBoost, and additionally compared against prompting-based large language models (LLMs) and thresholding techniques. They report: Among the evaluated machine learning models, Random Forest achieved the best overall performance on the TU/e Computer Science dataset and was therefore selected as the classifier used in CourseGraph.

For the intra-university evaluation on the TU/e dataset, the authors compared: (1) similarity thresholds (global and per-component), (2) classical machine learning models (Logistic Regression, Random Forest, XGBoost), and (3) prompting-based LLMs (ChatGPT-5.4, ChatGPT-5.4-mini, DeepSeek-V3).

Results reported in Table I show: "threshold-based methods provide a reasonable baseline. Among the individual components, course descriptions perform best (F1 = 0.69), followed closely by LOs (F1 = 0.61), while prerequisites produce many false positives. The zero-shot LLMs achieve relatively high precision but very low recall, making them too conservative for our application, where missing overlapping courses is more costly than false positives. The supervised NLP methods perform best overall. Random Forest and XGBoost provide the best balance between precision and recall, with F1 scores of 0.74 and 0.73, respectively."

Specifically, Random Forest achieved: Precision 0.78 (±0.03), Recall 0.80 (±0.12), F1 score 0.74 (±0.05), Accuracy 0.84 (±0.03). XGBoost achieved: Precision 0.68 (±0.06), Recall 0.85 (±0.03), F1 score 0.73 (±0.07), Accuracy 0.84 (±0.03). The authors note that missing overlapping courses is more costly than false positives, which favors the higher-recall supervised methods.

For the international mobility evaluation, the former program director provided "a dataset of 33 decisions made for real students that went to other universities on an Erasmus+ grant. After removing duplicates and courses with missing information, 24 decisions remain. In total, the dataset consists of 5 students' programs, spread over 3 universities (TU Delft, University of California - Santa Cruz, TU München)."

For the first student (at TU Delft), Figure 3 visualizes the embedding space using t-SNE projection. The authors find that Distributed Data Systems gets assigned to Distributed Systems (EDAP25), which corresponds with the annotation of the program director. However, "Research in Cyber Security - Hacking Lab and Systems Security both get assigned to Secure Systems Engineering (EITP20). This is probably because for now, CourseGraph does not factor in differences in learning methods yet, we just focus on the content of the course, expressed in LOs, course descriptions and topics. The authors note that the Hacking lab course is much more applied. Such projects, where students have to create new solutions using knowledge they have previously acquired correspond to higher levels of cognitive understanding according to the Bloom taxonomy. CourseGraph is also not fully correct in its matchings: Error Correcting Codes, mapped to Channel Coding for Reliable Communication (EITN70) by the program director, was mapped to Communication Systems (EITA55) by CourseGraph, since Communication Systems is a basic first-year course while the other two are advanced courses."

The authors emphasize CourseGraph's interpretability with a concrete example comparing Distributed Data Systems (TUD) and Distributed Systems (LU). Table II shows cosine similarities of 0.70 for descriptions and 0.81 for learning objectives. The authors explain: "We see that the LOs contributed the most to this particular decision. Upon inspecting the data, this makes sense: The description of the course at TUD is a lot longer and contains a more marketing-focused tone. For both courses, the LOs are most clearly defined and upon comparison, overlap the most. This is also reflected in the cosine similarity score, which is significantly higher for the LOs than for the other course components."

The authors acknowledge several assumptions and limitations: "the proposed methodology relies on several assumptions that may limit its applicability across institutions. Most notably, it assumes that all course information is publicly available and can be compared between institutions. They note that the academic system itself can differ considerably across institutions. For example, even if two universities both use the words 'introductory' and 'advanced' to indicate the level of a course, their definitions of these terms may vary. Additionally, the amount of publicly available information tends to differ between institutions. Since learning outcomes were found to be informative feature, programs that do not specify course-wise intended learning outcomes are likely to yield less reliable predictions."

They also note that the binary classification formulation "was primarily driven by the limited availability of labeled data. While distinguishing between full and partial overlap would better reflect the complexity of course equivalence and mimic the decision process of the program director, this would also require annotations of the extent to which two courses overlap. This information was unavailable for the TU/e CS dataset and could therefore not be incorporated in finetuning CourseGraph."

Future work…

Using CourseGraph's methodology and findings, here are the concrete improvements to make and the resulting AI system capabilities:

Improvements I can make:

1. Adopt a component-aware semantic representation – Represent each course as multiple BERT embeddings: separate embeddings for title, description sentences, prerequisites, level, and each individual learning outcome (LO), rather than a single document embedding. This allows fine-grained, interpretable comparisons.

2. Extract size-invariant similarity features – From the pairwise similarity matrices between components (e.g., LOs, sentences), compute maximum, minimum, mean, and fractions above thresholds (0.5, 0.6, 0.7) so the classifier works regardless of course length.

3. Train a Robust Random Forest classifier – Use labeled course pairs (expert-approved overlaps as positives, random pairs as negatives) with balanced precision/recall objectives, prioritizing recall since missing an overlap is costlier than a false positive in academic credit transfer.

4. Provide per-component explainability – Output which course components (description, LOs, title, prerequisites, level) contributed most to the overlap score, and visualize the course embedding space (e.g., t-SNE) to show nearest-neighbor matches.

5. Handle heterogeneous course webpages with LLM-based preprocessing – Automatically scrape and standardize course information (title, description, prerequisites, LOs, level) from public webpages using an LLM like DeepSeek-V3, with fallback handling for missing fields.

6. Enhance classification with contextual metadata – Include course level (G1/G2/A) and prerequisite indicators as features to improve distinction between introductory and advanced courses, addressing a known failure mode (e.g., confusing first-year courses with advanced ones).

7. Extend to multilingual course catalogs – Replace sBERT with multilingual sentence embeddings (e.g., paraphrase-multilingual-MiniLM or XLM-R) to compare courses offered in different languages.

8. Add ordinal overlap categories – Move from binary classification to multi-class (no overlap, partial overlap, full equivalence) when annotation data with overlap percentages is available, better matching the 20%/66.7% thresholds used by evaluators.

9. Integrate pedagogical and assessment factors – Incorporate learning activities, examination formats, and Bloom's taxonomy levels from the course pages to capture differences in applied vs. conceptual content, reducing false matches where one course is a hands-on lab and the other is theoretical.

10. Use cross-component relationships – Relate prerequisites of one course to learning outcomes of another, and compare LOs across components (e.g., prerequisites ↔ LOs) to improve matching when descriptions are marketing-heavy or vague.

The improved AI system can:

• Automatically evaluate whether a course taken abroad overlaps with a home-institution course, producing a decision (overlap / no overlap) with a confidence score.

• Show a breakdown of the overlap decision: e.g., 72% similarity in learning outcomes, 45% similarity in descriptions, 60% in prerequisites so students and advisors can verify the reasoning.

• Visualize the semantic space of courses, clustering related courses and highlighting nearest neighbors, enabling exploratory search for equivalences.

• Distinguish between full equivalence and partial overlap (e.g., worth 2–5 ECTS vs. full credit transfer), mimicking the program director's decision process.

• Process course pages from different universities, in different languages, and with varying levels of detail, using multilingual embeddings and LLM-based field extraction.

• Flexibly adapt to both intra-university specialization tracks and international mobility (Erasmus+) by retraining on institution-specific expert annotations.

• Flag cases where course content matches but pedagogical approach differs (e.g., project-based vs. theoretical), prompting further human review.

• Scale to large catalogs, allowing students and advisors to screen hundreds of external courses in minutes instead of manual per-course analysis.

Student mobility programs such as Erasmus+ enable students to take courses at other universities, broadening their academic and cultural horizons. However, this flexibility also leads to a practical challenge: ensuring that students do not take courses elsewhere that substantially overlap with courses in their home curriculum. In this work, we propose CourseGraph, a methodology that automates the evaluation of external courses based on insights obtained from the process followed by curriculum administrators when assessing courses for inclusion in a degree program. Course- Graph extracts information such as course titles, descriptions, and learning outcomes from the course webpage. Then, this information is represented semantically using a BERT-based language model, after which the pair-wise similarity between courses can be computed. This information is then used by a Random Forest classifier to determine whether a candidate course abroad overlaps with a course already contained in the student's curriculum. We evaluate CourseGraph using (1) the Computer Science program at Eindhoven University of Technology, which contains information about courses with substantial overlap, and (2) six approved international programs from students enrolled in the Computer Science program at Lund University, including the corresponding decisions made by a curriculum administrator. The experimental results indicate that CourseGraph provides an effective approach for identifying overlapping courses and supporting curriculum alignment across universities.