An introduction to data distillation

Distillation ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. This is a kind of omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning offers the potential to surpass state-of-the-art fully supervised methods.

  • Introduction
  • Data Distillation
  • Data Distillation for Keypoint Detection
  • Experiments on Keypoint Detection
  • Thesis Work


  • Data Distillation: Towards Omni-Supervised Learning, Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, Kaiming He. arXiv:1712.04440 [cs] (2017)

In this series