Cold-Start Classification Task
A Cold-Start Classification Task is a classification task that aims to accurately classify instances belonging to classes with little or no labeled training data.
- AKA: Zero-Shot Classification, Few-Shot Classification, Low-Resource Classification, Cold-Start Learning.
- Context:
- Task Input: Unlabeled instances from unseen or rarely seen classes.
- Optional Input: Auxiliary information such as class descriptions, metadata, or labeled data from related tasks or domains.
- Task Output: Predicted class labels for the input instances.
- Task Performance Measures: Accuracy, F1-score, Precision, Recall, Area Under the ROC Curve (AUC).
- Task Objective: To generalize classification capabilities to new or underrepresented classes with minimal or no labeled examples.
- It can be systematically solved and automated by a Cold-Start Classification System.
- It can leverage transfer learning to apply knowledge learned from high-resource domains or tasks.
- It can utilize meta-learning strategies to quickly adapt to new classes with few examples.
- It can implement zero-shot learning using semantic information about target classes.
- It can involve data augmentation to synthetically expand training data for rare classes.
- It can incorporate self-supervised learning to learn general representations from unlabeled data.
- It can apply to practical scenarios such as medical diagnosis, intent classification in dialogue systems, and product categorization in e-commerce.
- It can be critical for real-world AI systems operating in dynamic environments with continuously emerging categories.
- ...
- Task Input: Unlabeled instances from unseen or rarely seen classes.
- Example(s):
- Classifying new product types in online retail with only product descriptions.
- Detecting rare disease categories in medical reports with minimal labeled samples.
- Intent classification in chatbots where newly introduced intents lack training data.
- ...
- Counter-Example(s):
- Standard Classification Task, which assumes a sufficient amount of labeled training data for all classes.
- Cold-Start Recommendation Task, which focuses on user-item recommendation rather than label prediction.
- Unsupervised Clustering Task, which groups data without using any class labels.
- ...
- See: Cold-Start Classification System, Zero-Shot Learning, Few-Shot Learning, Meta-Learning, Transfer Learning, Cold-Start Estimation Task.
References
2023a
- (Shnarch et al., 2023) ⇒ Eyal Shnarch, Ariel Gera, Alon Halfon, Lena Dankin, Leshem Choshen, Ranit Aharonov, & Noam Slonim. (2023). "Cluster & Tune: Boost Cold Start Performance in Text Classification".
- QUOTE: "Cluster & Tune proposes an intermediate unsupervised clustering phase between pretraining and fine-tuning of pretrained language models such as BERT to address the cold start problem in text classification. The method clusters unlabeled training data and uses these clusters as pseudo-labels for an intermediate classification task, significantly improving performance when labeled data is scarce, especially for topical classification tasks."
"Extensive experimental results demonstrate the practical value of this strategy on a variety of benchmark datasets. ... It is most prominently valuable when the training data available for the target task is relatively small and the classification task is of a topical nature."
"Inter-training with respect to sIB clusters consistently led to better results in the final performance on the target task, compared to inter-training with respect to the clusters obtained with K-means."
- QUOTE: "Cluster & Tune proposes an intermediate unsupervised clustering phase between pretraining and fine-tuning of pretrained language models such as BERT to address the cold start problem in text classification. The method clusters unlabeled training data and uses these clusters as pseudo-labels for an intermediate classification task, significantly improving performance when labeled data is scarce, especially for topical classification tasks."
2023b
- (Wang et al., 2023) ⇒ Shuang Wang, Yongchao Jin, Yong Liu, Jianxun Lian, Fuzheng Zhang, Xing Xie, & Guangzhong Sun. (2023). "A novel hybrid recommendation framework with cold-start capability based on ensemble learning". In: Information Sciences.
- QUOTE: "A hybrid recommendation framework is proposed for cold-start problems, combining ensemble learning and feature augmentation to improve recommendation accuracy for new users and items. The approach leverages both content-based and collaborative filtering techniques, showing significant gains over traditional methods in cold-start scenarios."
2022a
- (Papers with Code, 2022) ⇒ Papers with Code. (2022). "Semi-Supervised Image Classification (Cold Start)".
- QUOTE: "Semi-supervised image classification (cold start) refers to the challenge of classifying images when there are very few labeled examples available. Benchmarks and leaderboards track advances in methods that combine labeled and unlabeled data to improve performance under cold start conditions."
2022b
- (OpenReview, 2022) ⇒ OpenReview Authors. (2022). "Cold-Start Semi-Supervised Learning with Self-Supervised Pre-Training".
- QUOTE: "We study cold-start semi-supervised learning, where only a handful of labeled data are available at the beginning. Our approach leverages self-supervised pre-training to provide robust initial representations, followed by semi-supervised fine-tuning, achieving state-of-the-art results on several benchmarks under cold start conditions."