Cross-Domain Transfer Learning Benchmarking Task

A Cross-Domain Transfer Learning Benchmarking Task is an ML benchmarking task that is an evaluation framework designed to assess the performance of automated cross-domain transfer learning systems.

• AKA: Cross-Domain Transfer Learning Evaluation, Cross-Domain Transfer Learning Assessment, Cross-Domain Transfer Learning Benchmark.
• Context:
  • Task Input: Labeled data from a source domain and unlabeled or sparsely labeled data from a distinct target domain.
    • Optional Input: Domain-specific metadata, auxiliary tasks, or pre-trained models.
  • Task Output: Model performance metrics on the target domain, such as classification accuracy or prediction error.
  • System/Model Performance Measure: Accuracy, F1 score, AUC, transferability metrics like OTCE, and domain divergence measures.
  • Benchmark Datasets: Office-31, DomainNet, CD-FSL, NineRec.
  • It can evaluate the effectiveness of transfer learning algorithms in adapting knowledge from a source domain to a target domain with different distributions.
  • It can help identify the robustness and generalization capabilities of models across diverse domains.
  • It can facilitate the comparison of various transfer learning approaches under standardized conditions.
  • It can aid in understanding the limitations and challenges associated with cross-domain knowledge transfer.
  • ...
• Example(s):
  • CD-FSL Benchmark, which assesses few-shot learning models across diverse domains like medical and satellite imagery.
  • OTCE Metric, which evaluates transferability between source and target tasks using optimal transport-based conditional entropy.
  • NineRec Benchmark, a multimodal dataset for evaluating cross-domain recommendation systems.
  • ...
• Counter-Example(s):
  • Intra-Domain Transfer Learning Benchmarks which focuses solely on within-domain transfer learning without domain shifts.
  • Evaluation tasks that do not involve knowledge transfer between distinct domains.
  • ...
• See: Cross-Domain Transfer Learning, Adversarial Domain Adaptation, Transfer Learning, Few-Shot Learning, Meta-Learning, Automated Domain-Specific Writing Task, Domain-Specific Natural Language Generation Task, Domain-Specific Text Understanding Task.

---

---

References

2025

• (NineRec, 2025) ⇒ NineRec Team. (2025). "NineRec: Benchmark for Cross-Domain Recommendation". Retrieved: 2025-05-04.
  • QUOTE: "NineRec is a benchmark dataset for cross-domain recommendation, supporting the evaluation of recommendation algorithms that transfer knowledge between different domains."

2024

• (Tan et al., 2024) ⇒ Yang Tan, Yang Li, & Shao-Lun Huang. (2024). "Transferability-Guided Cross-Domain Cross-Task Transfer Learning".
  • QUOTE: "We propose a transferability metric called Optimal Transport based Conditional Entropy (OTCE), to analytically predict the transfer performance for supervised classification tasks in cross-domain and cross-task feature transfer settings. Our OTCE score characterizes transferability as a combination of domain difference and task difference, and explicitly evaluates them from data in a unified framework. Specifically, we use optimal transport to estimate domain difference and the optimal coupling between source and target distributions, which is then used to derive the conditional entropy of the target task (task difference). Experiments on the largest cross-domain dataset DomainNet and Office31 demonstrate that OTCE shows an average of 21% gain in the correlation with the ground truth transfer accuracy compared to state-of-the-art methods."

2021a

• (Zhu et al., 2021) ⇒ Zheng Zhu, Jianqiang Huang, Jingwei Zhang, Xiaoyong Shen, Jia Li, & Xiaolin Hu. (2021). "Transferable Feature Representation for Visual Domain Adaptation".
  • QUOTE: "We propose a transferable feature representation learning framework for visual domain adaptation, which aims to reduce the domain shift between the source domain and target domain by learning domain-invariant features. The approach achieves improved adaptation performance on standard domain adaptation benchmarks."

2021b

• (Zhuang et al., 2021) ⇒ Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hui Xiong, Qing He, & Zhi-Hua Zhou. (2021). "A Comprehensive Survey on Transfer Learning".
  • QUOTE: "Transfer learning aims to improve learning performance in a target domain by leveraging knowledge from a source domain. The survey covers various transfer learning scenarios, including cross-domain, cross-task, and cross-modal transfer learning, and discusses challenges such as domain discrepancy, negative transfer, and transferability measurement."

2020

• (Guo et al., 2020) ⇒ Yunhui Guo, Noah Golmant, Xinghua Lou, Vincent Dumoulin, Yann LeCun, Pietro Perona, Serge Belongie, Yong Jae Lee, Judy Hoffman, Eric M. Rudd, Kate Saenko, Trevor Darrell, & Ser-Nam Lim. (2020). "A Broader Study of Cross-Domain Few-Shot Learning".
  • QUOTE: "This paper proposes the Broader Study of Cross-Domain Few-Shot Learning (BSCD-FSL) benchmark, consisting of image data from a diverse assortment of image acquisition methods, including natural images, satellite images, dermatology images, and radiology images. ... Accuracy of all methods tend to correlate with dataset similarity to natural images, verifying the value of the benchmark to better represent the diversity of data seen in practice and guiding future research."

2007

• (Taylor & Stone, 2007) ⇒ Matthew E. Taylor & Peter Stone. (2007). "Cross-Domain Transfer for Reinforcement Learning".
  • QUOTE: "This paper investigates cross-domain transfer in reinforcement learning, where knowledge learned in one domain is transferred to a different, but related, domain. The results show that transfer learning can significantly reduce the number of learning episodes required in the target domain."

---