Include BYOD environment variables in custom endpoint

[cecheta]

Motivation

There are a number of env vars used when using Azure OpenAI On Your Data in the BYOD endpoint, which are not used when calling the custom endpoint, such as top p and temperature:

chat-with-your-data-solution-accelerator/code/create_app.py

Lines 96 to 160 in 54723fe

	response = openai_client.chat.completions.create(
	model=env_helper.AZURE_OPENAI_MODEL,
	messages=messages,
	temperature=float(env_helper.AZURE_OPENAI_TEMPERATURE),
	max_tokens=int(env_helper.AZURE_OPENAI_MAX_TOKENS),
	top_p=float(env_helper.AZURE_OPENAI_TOP_P),
	stop=(
	env_helper.AZURE_OPENAI_STOP_SEQUENCE.split("|")
	if env_helper.AZURE_OPENAI_STOP_SEQUENCE
	else None
	),
	stream=env_helper.SHOULD_STREAM,
	extra_body={
	"data_sources": [
	{
	"type": "azure_search",
	"parameters": {
	"authentication": (
	{
	"type": "api_key",
	"key": env_helper.AZURE_SEARCH_KEY,
	}
	if env_helper.is_auth_type_keys()
	else {
	"type": "system_assigned_managed_identity",
	}
	),
	"endpoint": env_helper.AZURE_SEARCH_SERVICE,
	"index_name": env_helper.AZURE_SEARCH_INDEX,
	"fields_mapping": {
	"content_fields": (
	env_helper.AZURE_SEARCH_CONTENT_COLUMNS.split("|")
	if env_helper.AZURE_SEARCH_CONTENT_COLUMNS
	else []
	),
	"title_field": env_helper.AZURE_SEARCH_TITLE_COLUMN or None,
	"url_field": env_helper.AZURE_SEARCH_URL_COLUMN or None,
	"filepath_field": (
	env_helper.AZURE_SEARCH_FILENAME_COLUMN or None
	),
	},
	"filter": env_helper.AZURE_SEARCH_FILTER,
	"in_scope": env_helper.AZURE_SEARCH_ENABLE_IN_DOMAIN,
	"top_n_documents": env_helper.AZURE_SEARCH_TOP_K,
	"embedding_dependency": {
	"type": "deployment_name",
	"deployment_name": env_helper.AZURE_OPENAI_EMBEDDING_MODEL,
	},
	"query_type": (
	"vector_semantic_hybrid"
	if env_helper.AZURE_SEARCH_USE_SEMANTIC_SEARCH
	else "vector_simple_hybrid"
	),
	"semantic_configuration": (
	env_helper.AZURE_SEARCH_SEMANTIC_SEARCH_CONFIG
	if env_helper.AZURE_SEARCH_USE_SEMANTIC_SEARCH
	and env_helper.AZURE_SEARCH_SEMANTIC_SEARCH_CONFIG
	else ""
	),
	"role_information": env_helper.AZURE_OPENAI_SYSTEM_MESSAGE,
	},
	}
	]
	},
	)

These should be added to the custom endpoint, to create a consistent experience when switching between the two endpoints.

Requirements

A list of requirements to consider this feature delivered

• Add all relevant Azure OpenAI env vars from the BYOD endpoint to the custom endpoint

Tasks

To be filled in by the engineer picking up the issue

[github-actions]

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 30 days.