CoRCi: Cross-Reconstruction of Coherent Interests Modeling in Cross-Domain Sequential Recommendation

CoRCi: Cross-Reconstruction of Coherent Interests Modeling in Cross-Domain Sequential Recommendation proposes a dual-target Cross-Domain Sequential Recommendation (CDSR) framework to address limitations in existing methods that construct mixed-domain sequences by merging domain-specific sequences chronologically. The paper identifies that current CDSR models typically deploy separate encoders for mixed-domain sequences and train them with per-domain loss aggregation, which magnifies inter-domain discrepancies and disrupts domain-invariant interest coherence, especially when query–target pairs in Seq2Seq originate from different domains.

To tackle these drawbacks, CoRCi introduces two key innovations. First, it proposes a Cross-Reconstruction approach that generates mixed-domain representations directly from pre-encoded specific-domain representations via cross-attention. Specifically, the Cross-Reconstruction module (CRM) uses cross-attention with the mixed-domain embedding as the query and the sum of the two specific-domain representations as keys and values, reconnecting coherent domain-invariant interests that are otherwise dispersed across specific-domain sequences. Second, CoRCi introduces FocalNCE, which embeds Focal Loss into the mixed-domain InfoNCE objective in a domain-agnostic manner. FocalNCE assigns higher penalties to negatives drawn from the same domain as the query, thereby strengthening domain-invariant alignment and mitigating intra-domain bias.

The model architecture consists of: (1) sequence formation with shared item and positional embedding layers; (2) separate self-attention encoders (SAE) for each specific domain; (3) the Cross-Reconstruction module (CRM) that produces mixed-domain representations; (4) Cross-Attention Transferors (CAT) that transfer mixed-domain knowledge back to specific-domain representations using cross-attention with a stop-gradient operation on the mixed-domain representations; and (5) a loss function combining per-domain InfoNCE losses (with the mixed-domain representation added as a task-agnostic backbone) and the mixed-domain FocalNCE loss weighted by a hyperparameter β.

Extensive experiments are conducted on four real-world datasets: three two-domain datasets from Amazon (Food-Kitchen, Beauty-Electronics, Movie-Book) and one three-domain dataset from Douban (Movie-Book-Music) with non-overlapping users. The results show that CoRCi consistently outperforms state-of-the-art CDSR counterparts, achieving statistically significant gains across all metrics (paired t-tests, p ≤ 0.01). The average per-domain gains across four metrics are 4.99% and 6.33% on AFK, 5.77% and 11.06% on ABE, and 3.18% and 4.68% on AMB.

Ablation studies confirm the effectiveness of each component: using one shared encoder (V1enc) or three separate encoders (V3enc) underperforms the proposed SAE-CR hybrid; removing the cross-attention transferors (Vw/o-cat) degrades performance; removing the stop-gradient operation (Vw/o-sg) or the mixed-domain task (Vw/o-LM) also leads to marked declines. Loss selection experiments show that using FocalNCE only for the mixed-domain loss (as in CoRCi) outperforms alternatives such as applying FocalNCE to all losses (Vff), InfoNCE to all losses (Vii), or computing the mixed-domain loss separately per domain (Vdt). The focusing parameter α is tuned per dataset, with optimal values of α=4 for AFK and α=2 for ABE and AMB. The weight β of the mixed-domain loss shows a concave relationship with performance, with optima at β=1.0 for AFK and ABE and β=0.7 for AMB. On the three-domain Douban dataset, CoRCi achieves significant gains over the best baseline ABXI in all nine domain-metric combinations (p < 0.01). A case study further demonstrates CoRCi's ability to preserve coherent domain-invariant interests and deliver accurate recommendations.

Improvements to AI systems:

1. Cross-Reconstruction Module (CRM) for Multi-Domain Representation Learning

• Improvement: Replace naive sequence concatenation with cross-attention that uses mixed-domain embeddings as queries and summed domain-specific representations as keys/values. This directly reconstructs coherent cross-domain interests without chronological merging.

• Improved system capability: AI can now fuse information from multiple domains while preserving domain-invariant patterns, reducing interference from domain-specific noise. This is especially useful for recommendation systems, user profiling, and any multi-source sequential data (e.g., browsing + purchase history).

2. FocalNCE Loss for Domain-Agnostic Contrastive Learning

• Improvement: Integrate focal loss weighting into InfoNCE, where negatives from the same domain as the query receive higher penalties. This dynamically reweights hard negatives to strengthen domain-invariant alignment.

• Improved system capability: AI systems can now learn more robust representations that are less biased toward a single domain, improving generalization across tasks with imbalanced or heterogeneous data sources (e.g., cross-platform user behavior modeling).

3. Hybrid Encoder Architecture (SAE + CRM + CAT)

• Improvement: Use separate self-attention encoders per domain, then a cross-reconstruction module for mixed-domain representation, and cross-attention transferors with stop-gradient to propagate mixed-domain knowledge back to specific-domain encoders.

• Improved system capability: AI can now maintain domain-specific fidelity while leveraging shared knowledge, avoiding catastrophic forgetting or over-mixing. This enables better performance in multi-task learning, transfer learning, and sequential decision-making across related but distinct environments.

4. Task-Agnostic Mixed-Domain Backbone with Per-Domain Losses

• Improvement: Add the mixed-domain representation as an auxiliary backbone to per-domain InfoNCE losses, weighted by a tunable hyperparameter β. This decouples domain-specific optimization from global coherence.

• Improved system capability: AI systems can now balance local and global objectives, leading to more stable training and better convergence in multi-domain or multi-objective settings (e.g., recommendation with multiple user intents, robotics with multiple sensor modalities).

5. Stop-Gradient Operation in Cross-Attention Transfer

• Improvement: Prevent gradient backpropagation from specific-domain encoders into the mixed-domain representation during transfer, avoiding representation drift.

• Improved system capability: AI can now perform knowledge distillation or cross-modal transfer without destabilizing the shared representation, enabling more reliable continual learning and domain adaptation.

What the improved AI system can do:

• Recommend items across multiple domains (e.g., e-commerce, streaming, social media) with higher accuracy and coherence, especially when user interests are fragmented across domains.

• Learn user representations that are invariant to domain shifts, improving cold-start performance in new domains.

• Handle three or more domains simultaneously without performance degradation, as demonstrated on the Douban dataset.

• Provide interpretable cross-domain interest alignment, as shown in case studies, enabling explainable recommendations.

• Achieve statistically significant gains (4–11% relative improvement) over state-of-the-art baselines in real-world benchmarks, with robust hyperparameter tuning (α, β) per dataset.

Cross-Domain Sequential Recommendation (CDSR) aims to alleviate data sparsity by transferring dynamic user interests across related domains. A key challenge lies in effectively bridging these domains. In single-domain modeling, models cannot distinguish between domain-specific and domain-invariant interests. Recent methods merge domain-specific sequences chronologically into a mixed-domain sequence to capture domain-invariant knowledge. However, they typically deploy separate encoders for the mixed-domain sequence and train them with per-domain loss aggregation. This workflow magnifies inter-domain discrepancies and disrupts domain-invariant interest coherence, especially when query target pairs in Seq2Seq originate from different domains. In this paper, we present CoRCi (Cross-Reconstruction for Coherent Interest), a dual-target CDSR framework that tackles these drawbacks. Specifically, CoRCi proposes a Cross-Reconstruction approach that generates mixed-domain representations directly from pre-encoded specific-domain representations via cross-attention. The generated representations are then trained using a single, sequence-level, domain-agnostic loss to preserve the coherence of domain-invariant interests. To further suppress domain discrepancies in mixed-domain modeling, CoRCi introduces FocalNCE, which embeds Focal Loss into the preceding mixed-domain InfoNCE objective. The new loss assigns higher penalties to negatives drawn from the same domain as the query, thereby strengthening domain-invariant alignment. Extensive experiments on four real-world datasets demonstrate that CoRCi consistently outperforms state-of-the-art CDSR counterparts, achieving statistically significant gains across all metrics.