Project Background:
Self-supervised clustering is a technique within unsupervised learning that leverages the inherent structure of data to learn meaningful representations without requiring explicit labels. Unlike traditional clustering methods, self-supervised clustering does not rely on external annotations or ground truth labels for training. Instead, it generates pseudo-labels or targets from the data itself through pretext tasks or auxiliary objectives. These pretext tasks are designed to encourage the model to capture salient features or underlying patterns in the data, then be used for downstream tasks such as classification or clustering.
Common pretext tasks in self-supervised clustering include predicting missing or corrupted parts of the input data, generating contextually relevant representations, or learning to distinguish between different views or transformations of the same data instance. By learning representations in a self-supervised manner, the model can capture high-level semantic information and structure in the data, facilitating more effective clustering and downstream tasks.
Self-supervised clustering has shown promising results in various domains, including computer vision, natural language processing, and biomedical data analysis, where labeled data may be scarce or expensive to obtain. It offers a data-driven approach to learning representations that can capture complex relationships and patterns in the absence of labeled supervision, thereby enabling more robust and scalable solutions for unsupervised learning tasks.
Problem Statement
- Obtaining ground truth labels for clustering tasks often requires manual annotation by domain experts, which is time-consuming and labor-intensive.
-
Ground truth labels for clustering tasks may be ambiguous or subjective, leading to challenges in training accurate clustering models.
- Clustering tasks often involve heterogeneous data sources or modalities, making it difficult to define a uniform labeling scheme or ground truth representation.
- Traditional clustering methods may struggle to scale to large datasets or high-dimensional feature spaces, limiting their applicability to real-world big data scenarios.
- Clustering models trained on labeled data may not generalize well to new or unseen data distributions, leading to poor performance in real-world deployment scenarios.
-
Clustering algorithms may be sensitive to noisy or irrelevant features in the data, leading to suboptimal cluster assignments and reduced clustering performance.
Aim and Objectives
- To develop self-supervised clustering techniques that can learn meaningful representations from unlabeled data without requiring external supervision.
- Design pretext tasks or auxiliary objectives that encourage the model to capture salient features and underlying patterns in the data.
- Develop algorithms for generating pseudo-labels or targets from the data itself to facilitate clustering.
- Investigate methods for learning representations that capture high-level semantic information and structure in the absence of labeled supervision.
- Evaluate the effectiveness of self-supervised clustering techniques on various unsupervised learning tasks and datasets.
- Explore applications of self-supervised clustering in domains where labeled data may be scarce or expensive to obtain.
Contributions to Self-supervised Clustering
- Development of pretext tasks and auxiliary objectives for learning meaningful representations from unlabeled data.
- Design of algorithms for generating pseudo-labels or targets from the data itself to facilitate clustering.
- Exploration of methods for capturing high-level semantic information and structure in the absence of labeled supervision.
- Empirical validation of the effectiveness of self-supervised clustering techniques on various unsupervised learning tasks and datasets.
- Advancement of applications of self-supervised clustering in domains where labeled data may be scarce or expensive to obtain.
Deep Learning Algorithms for Self-supervised Clustering
- Deep Embedded Clustering (DEC)
-
Deep Adaptive Feature Clustering (DAFC)
-
Deep Convolutional Embedded Clustering (DCEC)
-
Deep Self-Training for Clustering (DSTC)
-
Deep Generative Clustering (DGC)
-
Deep Unsupervised Clustering via Bayesian Nonparametrics (DUCBN)
-
Deep Unsupervised Clustering via Variational Autoencoder (DUCVAE)
-
Deep Graph Clustering (DGC)
-
Deep Reinforcement Learning for Clustering (DRLC)
-
Deep Self-supervised Clustering (DSC)
Datasets for Self-supervised Clustering
- MNIST
-
CIFAR-10
-
ImageNet
-
Fashion-MNIST
-
COCO (Common Objects in Context)
-
LSUN (Large-Scale Scene Understanding)
-
LFW (Labeled Faces in the Wild)
-
CelebA
-
SVHN (Street View House Numbers)
-
OpenAI Gym environments
Software Tools and Technologies
Operating System: Ubuntu 18.04 LTS 64bit / Windows 10
Development Tools: Anaconda3, Spyder 5.0, Jupyter Notebook
Language Version: Python 3.9
Python Libraries:
1. Python ML Libraries:
- Scikit-Learn
- Numpy
- Pandas
- Matplotlib
- Seaborn
- Docker
- MLflow
2. Deep Learning Frameworks:
- Keras
- TensorFlow
- PyTorch
