Cross-Domain Recommendation Benchmarking Task

A Cross-Domain Recommendation Benchmarking Task is a NLP benchmarking task that provides a standardized evaluation setting to assess a cross-domain recommendation system's or model's performance on a specific task of cross-domain recommendation.

• AKA: Cross-Domain Recommendation Benchmark.
• Context:
  • Task Input: User-item interaction data spanning multiple domains (e.g., movies, books, music).
    • Optional Input: Item metadata (e.g., descriptions, images), user profiles, contextual information (e.g., timestamps, locations).
  • Task Output: Personalized item recommendations in the target domain.
  • System/Model Performance Measure: Metrics such as Precision@K, Recall@K, NDCG@K, Mean Reciprocal Rank (MRR), and Root Mean Square Error (RMSE).
  • Benchmark Datasets: NineRec, Tenrec, AmazonKG4CDR, Douban, Amazon 5-core.
  • It can evaluate a cross-domain recommendation system's ability to transfer knowledge from source domains to improve recommendations in a target domain.
  • It can assess performance in scenarios with varying degrees of domain overlap, including cold-start situations.
  • It can test the effectiveness of different transfer learning techniques, such as matrix factorization, deep learning, and graph-based methods.
  • It can provide insights into the generalization capabilities of recommendation models across diverse domains.
  • ...
• Example(s):
  • NineRec Benchmark, which evaluates transferable recommendation models across nine diverse target domains with multimodal data.
  • Tenrec Benchmark, a large-scale dataset encompassing multiple recommendation scenarios, including cross-domain and transfer learning tasks.
  • AmazonKG4CDR Benchmark, which incorporates knowledge graphs to enhance cross-domain recommendation evaluations.
  • ...
• Counter-Example(s):
  • Single-Domain Recommendation Benchmark, which evaluates models within a single domain without assessing cross-domain transfer capabilities.
  • General Recommendation Benchmark, which may not specifically focus on the challenges and nuances of cross-domain recommendation tasks.
  • Cold-Start Recommendation Benchmark, which targets new user/item scenarios without necessarily involving multiple domains.
  • ...
• See: Cross-Domain Recommendation Task, Domain-Specific Text Understanding Task, Cross-Domain Transfer Learning Task, Automated Domain-Specific Writing Task, Targeted Concept Simplification Task, Transfer Learning, Matrix Factorization, Graph Neural Networks.

---

---

References

2023

• (Zhao et al., 2023) ⇒ Ying Zhao, Yingqiang Ge, Xiaoyu Liu, Xiangyu Zhao, Fajie Yuan, & Xiangnan He. (2023). "Cross-domain Recommendation: Progress and Prospects". arXiv Preprint.
  • QUOTE: Cross-domain recommendation aims to leverage user-item interactions from source domains to improve recommendation performance in target domains. The survey summarizes recent advances in representation learning, transfer learning, and multi-task learning for cross-domain recommendation, and highlights challenges such as domain discrepancy, negative transfer, and data sparsity.

2022

• (Yuan et al., 2022) ⇒ Guanghu Yuan, Fajie Yuan, Yudong Li, Beibei Kong, Shujie Li, Lei Chen, Min Yang, Chenyun Yu, Bo Hu, Zang Li, Yu Xu, & Xiaohu Qie. (2022). "Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems". In: NeurIPS 2022 Datasets and Benchmarks Track.
  • QUOTE: Tenrec is a large-scale, multipurpose benchmark dataset for recommender systems, containing around 5 million users and 140 million interactions across four recommendation scenarios. It uniquely provides multiple types of user feedback (clicks, likes, shares, follows, reads, favorites) and includes both positive and true negative feedback, supporting cross-domain recommendation, multi-task learning, and CTR prediction. Tenrec enables evaluation of ten diverse recommendation tasks and features overlapped users/items for transfer learning research.

2022b

• (Wang et al., 2022) ⇒ Yixin Wang, Xiaoyan Yang, Yong Liu, Jianxun Lian, Fuzheng Zhang, & Xing Xie. (2022). "A Unified Framework for Cross-Domain and Cross-System Recommendation". arXiv Preprint.
  • QUOTE: We propose a unified framework for cross-domain recommendation and cross-system recommendation, which models user preference transfer and domain-specific features through shared representation learning. Experimental results on public benchmark datasets demonstrate 17.9% improvement in NDCG over single-domain baselines.

---