以图搜图如何使用自己的模型

[David1-git]

在
p_embed = (
pipe.input('src')
.flat_map('src', 'img_path', load_image)
.map('img_path', 'img', ops.image_decode())
.map('img', 'vec', ops.image_embedding.timm(model_name=MODEL, device=DEVICE))
)
现在我有一个行人重识别的模型，在这里我需要怎么样做，才能替换为自己的模型。非常期待您的回复，感谢！

[junjiejiangjjj]

If you want to pass in the weights of a PyTorch model, you can pass it in as parameter: ops.image_embedding.timm(model_name=MODEL, device=DEVICE, checkpoint_path=YOUR_WEIGHTS_PATH)
If it contains PyTorch code, a new operator needs to be created.
https://towhee.readthedocs.io/en/latest/operator/usage.html#run-pipeline-with-named-operators
You can refer to this implementation
https://towhee.io/image-embedding/timm/raw/branch/main/timm_image.py

[David1-git]

If you want to pass in the weights of a PyTorch model, you can pass it in as parameter: ops.image_embedding.timm(model_name=MODEL, device=DEVICE, checkpoint_path=YOUR_WEIGHTS_PATH) If it contains PyTorch code, a new operator needs to be created. https://towhee.readthedocs.io/en/latest/operator/usage.html#run-pipeline-with-named-operators You can refer to this implementation https://towhee.io/image-embedding/timm/raw/branch/main/timm_image.py

我可以使用timm库注册自己的模型吗

[David1-git]

If you want to pass in the weights of a PyTorch model, you can pass it in as parameter: ops.image_embedding.timm(model_name=MODEL, device=DEVICE, checkpoint_path=YOUR_WEIGHTS_PATH) If it contains PyTorch code, a new operator needs to be created. https://towhee.readthedocs.io/en/latest/operator/usage.html#run-pipeline-with-named-operators You can refer to this implementation https://towhee.io/image-embedding/timm/raw/branch/main/timm_image.py

我想要替换掉以图搜图里的模型，使用自己的行人重识别模型。换句话说，我想让我的行人重识别模型可以像以图搜图项目那样部署在服务器上

[junjiejiangjjj]

https://github.com/huggingface/pytorch-image-models/tree/main/timm/models
Can you find the model you are using there? If so, you can use timm.

[David1-git]

https://github.com/huggingface/pytorch-image-models/tree/main/timm/models Can you find the model you are using there? If so, you can use timm.

这个里面没有我使用的模型

[junjiejiangjjj]

Then you'll have to implement an op yourself.

[David1-git]

Then you'll have to implement an op yourself.
具体要怎么实现呢，可以具体说一下吗

[junjiejiangjjj]

Then you'll have to implement an op yourself.
具体要怎么实现呢，可以具体说一下吗

https://towhee.readthedocs.io/en/latest/operator/usage.html#run-pipeline-with-named-operators

[watertianyi]

@junjiejiangjjj
I found that there is ConvNeXt-V2 in timm, but I use https://github.com/open-mmlab/mmpretrain or https://github.com/facebookresearch/ConvNeXt-V2 for training. You can directly use timm to load this Is it a model trained by two warehouses?

[junjiejiangjjj]

@junjiejiangjjj I found that there is ConvNeXt-V2 in timm, but I use https://github.com/open-mmlab/mmpretrain or https://github.com/facebookresearch/ConvNeXt-V2 for training. You can directly use timm to load this Is it a model trained by two warehouses?

You can use checkpoint_path param to load your weights: ops.image_embedding.timm(model_name='resnet50', checkpoint_path="your weights file path")

[watertianyi]

@junjiejiangjjj

Why does the result graph when I run the same code look like this?
https://github.com/towhee-io/examples/blob/afbd207d9e65798d913b8f4eff8ba355000ada84/image/reverse_image_search/1_build_image_search_engine.ipynb

[junjiejiangjjj]

This op decodes the image in bgr format by default, so the displayed image color is incorrect. Use ops.image_decode.cv2('rgb') to display it normally.

[watertianyi]

@junjiejiangjjj Thank you for your prompt reply!