[Feature Request] Proposal to Allow Dynamic Use of Additional Embedding Models

[e7217]

Is your feature request related to a problem? Please describe.
Hello!

How about improving the system to allow the use of other embedding models dynamically, in addition to the predefined models?

AutoRAG/autorag/vectordb/base.py

Line 19 in 499f54a

self.embedding = embedding_models[embedding_model]()

AutoRAG/autorag/__init__.py

Lines 65 to 103 in 499f54a

	embedding_models = {
	# llama index
	"openai": LazyInit(
	OpenAIEmbedding
	), # default model is OpenAIEmbeddingModelType.TEXT_EMBED_ADA_002
	"openai_embed_3_large": LazyInit(
	OpenAIEmbedding, model_name=OpenAIEmbeddingModelType.TEXT_EMBED_3_LARGE
	),
	"openai_embed_3_small": LazyInit(
	OpenAIEmbedding, model_name=OpenAIEmbeddingModelType.TEXT_EMBED_3_SMALL
	),
	"mock": LazyInit(MockEmbeddingRandom, embed_dim=768),
	# langchain
	"openai_langchain": LazyInit(OpenAIEmbeddings),
	}
	try:
	# you can use your own model in this way.
	from llama_index.embeddings.huggingface import HuggingFaceEmbedding
	embedding_models["huggingface_baai_bge_small"] = LazyInit(
	HuggingFaceEmbedding, model_name="BAAI/bge-small-en-v1.5"
	)
	embedding_models["huggingface_cointegrated_rubert_tiny2"] = LazyInit(
	HuggingFaceEmbedding, model_name="cointegrated/rubert-tiny2"
	)
	embedding_models["huggingface_all_mpnet_base_v2"] = LazyInit(
	HuggingFaceEmbedding,
	model_name="sentence-transformers/all-mpnet-base-v2",
	max_length=512,
	)
	embedding_models["huggingface_bge_m3"] = LazyInit(
	HuggingFaceEmbedding, model_name="BAAI/bge-m3"
	)
	except ImportError:
	logger.info(
	"You are using API version of AutoRAG."
	"To use local version, run pip install 'AutoRAG[gpu]'"
	)

[vkehfdl1]

@e7217 Do you have any idea?
If you suggest any idea to config the embedding model, I think we can talk about how to make it.

[e7217]

@vkehfdl1 Thank you for your reply.
I will also consider a good solution for this.