Skip to content

Commit

Permalink
[TinyCLIP] Update PRETRAINING.md
Browse files Browse the repository at this point in the history
  • Loading branch information
wkcn authored Jan 14, 2024
1 parent b4c66ae commit 4a13c40
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions TinyCLIP/docs/PRETRAINING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ In this document, we introduce ***auto weight inheritance*** and ***manual weigh

:star: **[Notice]** Please replace the training data loader with the one loading LAION-400M or YFCC-15M.

Reference: [OpenCLIP Data](https://github.com/mlfoundations/open_clip?tab=readme-ov-file#data)

### Auto weight inheritance training
In this part, we compress OpenCLIP ViT-B/32 to 25% of origin size using three stages, where the model is compressed from 100% to 75%, from 75% to 50% and from 50% to 25% in the three stages, respectively.

Expand Down

0 comments on commit 4a13c40

Please sign in to comment.