Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

paper TipAdapter #475

Open
junxnone opened this issue Oct 9, 2024 · 0 comments
Open

paper TipAdapter #475

junxnone opened this issue Oct 9, 2024 · 0 comments

Comments

@junxnone
Copy link
Owner

junxnone commented Oct 9, 2024 •
edited
Loading

Tip-Adapter

  • 利用 CLIP，从少量样本训练集中构建键值缓存模型来作为适配器，结合 CLIP 的输出综合生成结果
  • 无需训练 - 非参数方式，训练的话更优
  • 推理: 测试图像特征与缓存键计算相似度，聚合缓存值形成预测，再与 CLIP 原始预测通过残差连接结合
    • 其中涉及计算查询 - 键亲和力及线性组合缓存值等操作，通过调整残差比率平衡两项预测。
  • Tip-Adapter-F: 当few-shot 数量增加时，性能不如CoOP & CLIP-Adapter,所以提出了 Tip-Adapter-F
    • 将缓存模型的键作为可学习参数并微调，冻结缓存值和 CLIP 的两个编码器

image

Reference
@junxnone junxnone changed the title x TipAdapter paper TipAdapter Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@junxnone