Current progress:

• General

  • Random color distortion (from SimCLR)
  • Random Gaussian blur (from SimCLR)
  • Random multi-crop (from SwAV)
  • InfoNCE loss
  • Momentum encoder
  • Selective Kernel (SK) convolution layer (used in SimCLR v2)
  • LARS optimizer
  • Consine annealing with linear warmup scheduler
  • Linear scheduler (for torch<=1.9)
  • CSV logger
  • TensorBoard logger
  • Checkpoint saving and restoring
  • Command line parameter parser
  • YAML parameter parser
  • PyTorch workflow encapsulation
  • Dedicated evaluation script
  • Detailed readme file
  • Data Parallel
  • Distributed Data Parallel
  • Global batch normalization
  • Shuffling batch normalization

• Supervised baseline

  • ResNet
  • ViT
  • CIFAR-10
  • CIFAR-100
  • ImageNet-1k

• Self-supervised baseline

  • SimCLR (contrastive, ResNet)
  • MoCo v2 (contrastive, ResNet)
  • MoCo v3 (contrastive, ViT)
  • BYOL (non-contrastive, ResNet)
  • DINO (self-distillation, ViT)

To be continued ...