Migrating model catalog tutorial to Azure AI Studio (now GA)

02-exploring-and-comparing-different-llms/README.md

@@ -4,9 +4,9 @@
 
 > *Click the image above to view video of this lesson*
 
-With the previous lesson, we have seen how Generative AI is changing the technology landscape, how Large Language Models (LLMs) work and how a business - like our startup - can apply them to their use cases and grow! In this chapter, we're looking to compare and contrast different types of large language models, LLMs to understand their pros and cons.
+With the previous lesson, we have seen how Generative AI is changing the technology landscape, how Large Language Models (LLMs) work and how a business - like our startup - can apply them to their use cases and grow! In this chapter, we're looking to compare and contrast different types of large language models (LLMs) to understand their pros and cons.
 
-The next step in our startup's journey is exploring the current landscape of Large Language Models (LLMs) and understanding which are suitable for our use case.
+The next step in our startup's journey is exploring the current landscape of LLMs and understanding which are suitable for our use case.
 
 ## Introduction
 
@@ -26,7 +26,7 @@ After completing this lesson, you will be able to:
 
 ## Understand different types of LLMs
 
-Large Language Models (LLMs) can have multiple categorizations based on their architecture, training data, and use case. Understanding these differences will help our startup select the right model for the scenario, and understand how to test, iterate, and improve performance.
+LLMs can have multiple categorizations based on their architecture, training data, and use case. Understanding these differences will help our startup select the right model for the scenario, and understand how to test, iterate, and improve performance.
 
 There are many different types of LLM models, your choice of model depends on what you aim to use them for, your data, how much you're ready to pay and more.
 
@@ -38,6 +38,8 @@ Depending on if you aim to use the models for text, audio, video, image generati
 
 - **Text generation**. Most models are trained on text generation and you have a large variety of choices from GPT-3.5 to GPT-4. They come at different costs with GPT-4 being the most expensive. It's worth looking into the [Azure OpenAI playground](https://oai.azure.com/portal/playground?WT.mc_id=academic-105485-koreyst) to evaluate which models best fit your needs in terms of capability and cost.
 
+- **Multi-modality**. If you're looking to handle multiple types of data in input and output, you might want to look into models like [gpt-4 turbo with vision or gpt-4o](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#gpt-4-and-gpt-4-turbo-models?WT.mc_id=academic-105485-koreyst) - the latest releases of OpenAI models - which are capable to combine natural language processing to visual understanding, enabling interactions through multi-modal interfaces.
+
 Selecting a model means you get some basic capabilities, that might not be enough however. Often you have company specific data that you somehow need to tell the LLM about. There are a few different choices on how to approach that, more on that in the upcoming sections.
 
 ### Foundation Models versus LLMs
@@ -106,26 +108,32 @@ Models are just the Neural Network, with the parameters, weights, and others. Al
 ## How to test and iterate with different models to understand performance on Azure
 
 Once our team has explored the current LLMs landscape and identified some good candidates for their scenarios, the next step is testing them on their data and on their workload. This is an iterative process, done by experiments and measures.
-Most of the models we mentioned in previous paragraphs (OpenAI models, open source models like Llama2, and Hugging Face transformers) are available in the [Foundation Models](https://learn.microsoft.com/azure/machine-learning/concept-foundation-models?WT.mc_id=academic-105485-koreyst) catalog in [Azure Machine Learning studio](https://ml.azure.com/?WT.mc_id=academic-105485-koreyst).
+Most of the models we mentioned in previous paragraphs (OpenAI models, open source models like Llama2, and Hugging Face transformers) are available in the [Model Catalog](https://learn.microsoft.com/azure/ai-studio/how-to/model-catalog-overview?WT.mc_id=academic-105485-koreyst) in [Azure AI Studio](https://ai.azure.com/?WT.mc_id=academic-105485-koreyst).
+
+[Azure AI Studio](https://learn.microsoft.com/azure/ai-studio/what-is-ai-studio?WT.mc_id=academic-105485-koreyst) is a Cloud Platform designed for developers to build generative AI applications and manage the whole development lifecycle - from experimentation to evaluation - by combining all Azure AI services into a single hub with an handy GUI. The Model Catalog in Azure AI Studio enables the user to:
+
+- Find the Foundation Model of interest in the catalog - either proprietary or open source, filtering by task, license, or name. To improve searchability, the models are organized into collections, like Azure OpenAI collection, Hugging Face collection, and more.
+
+![Model catalog](./images/AzureAIStudioModelCatalog.png?WT.mc_id=academic-105485-koreyst)
 
-[Azure Machine Learning](https://azure.microsoft.com/products/machine-learning/?WT.mc_id=academic-105485-koreyst) is a Cloud Service designed for data scientists and ML engineers to manage the whole ML lifecycle (train, test, deploy and handle MLOps) in a single platform. The Machine Learning studio offers a graphical user interface to this service and enables the user to:
+- Review the model card, including a detailed description of intended use and training data, code samples and evaluation results on internal evaluations library.
 
-- Find the Foundation Model of interest in the catalog, filtering by task, license, or name. It’s also possible to import new models that are not yet included in the catalog.
-- Review the model card, including a detailed description and code samples, and test it with the Sample Inference widget, by providing a sample prompt to test the result.
+![Model card](./images/ModelCard.png?WT.mc_id=academic-105485-koreyst)
 
-![Model card](./images/Llama1.png?WT.mc_id=academic-105485-koreyst)
+- Compare benchmarks across models and datasets available in the industry to assess which one meets the business scenario, through the [Model Benchmarks](https://learn.microsoft.com/azure/ai-studio/how-to/model-benchmarks?WT.mc_id=academic-105485-koreyst) pane.
 
-- Evaluate model performance with objective evaluation metrics on a specific workload and a specific set of data provided in input.
+![Model benchmarks](./images/ModelBenchmarks.png?WT.mc_id=academic-105485-koreyst)
 
-![Model evaluation](./images/Llama2.png?WT.mc_id=academic-105485-koreyst)
+- Fine-tune the model on custom training data to improve model performance in a specific workload, leveraging the experimentation and tracking capabilities of Azure AI Studio.
 
-- Fine-tune the model on custom training data to improve model performance in a specific workload, leveraging the experimentation and tracking capabilities of Azure Machine Learning.
+![Model fine-tuning](./images/FineTuning.png?WT.mc_id=academic-105485-koreyst)
 
-![Model fine-tuning](./images/Llama3.png?WT.mc_id=academic-105485-koreyst)
+- Deploy the original pre-trained model or the fine-tuned version to a remote real time inference - managed compute - or serverless api endpoint - [pay-as-you-go](https://learn.microsoft.com/azure/ai-studio/how-to/model-catalog-overview#model-deployment-managed-compute-and-serverless-api-pay-as-you-go?WT.mc_id=academic-105485-koreyst) - to enable applications to consume it.
 
-- Deploy the original pre-trained model or the fine-tuned version to a remote real time inference or batch endpoint, to enable applications to consume it.
+![Model deployment](./images/ModelDeploy.png?WT.mc_id=academic-105485-koreyst)
 
-![Model deployment](./images/Llama4.png?WT.mc_id=academic-105485-koreyst)
+> [!NOTE]
+> Not all models in the catalog are currently available for fine-tuning and/or pay-as-you-go deployment. Check the model card for details on the model's capabilities and limitations.
 
 ## Improving LLM results
 

18-fine-tuning/python/openai/oai-assignment.ipynb

@@ -54,7 +54,7 @@
     "Upload the data using the Files API [as described here](https://platform.openai.com/docs/guides/fine-tuning/upload-a-training-file). Note that in order to run this code, you must have done the following steps first:\n",
     " - Installed the `openai` Python package (make sure you use a version >=0.28.0 for latest features)\n",
     " - Set the `OPENAI_API_KEY` environment variable to your OpenAI API key\n",
-    "To learn more, see the [Setup guide](./../../../00-course-setup/SETUP.md) provided for the course.\n",
+    "To learn more, see the [Setup guide](./../../../00-course-setup/SETUP.md?WT.mc_id=academic-105485-koreyst) provided for the course.\n",
     "\n",
     "Now, run the code to create a file for upload from your local JSONL file."
    ]
@@ -264,7 +264,7 @@
    "source": [
     "You can also view the status by visiting the OpenAI website and exploring the _Fine-tuning_ section of the platform. This will show you the status of the current job, and also let you track the history of prior job execution runs. In this screenshot, you can see that the prior execution failed, and the second run succeeeded. For context, this happened when the first run used a JSON file with incorrectly formatted records  - once fixed, the second run completed successfully and made the model available for use.\n",
     "\n",
-    "![Fine-tuning job status](./img/fine-tuned-model-status.png)"
+    "![Fine-tuning job status](./img/fine-tuned-model-status.png?WT.mc_id=academic-105485-koreyst)"
    ]
   },
   {
@@ -275,7 +275,7 @@
     "\n",
     "| Messages | Metrics |\n",
     "|:---|:---|\n",
-    "| ![Messages](./img/fine-tuned-messages-panel.png) |  ![Metrics](./img/fine-tuned-metrics-panel.png)|\n"
+    "| ![Messages](./img/fine-tuned-messages-panel.png?WT.mc_id=academic-105485-koreyst) |  ![Metrics](./img/fine-tuned-metrics-panel.png?WT.mc_id=academic-105485-koreyst)|\n"
    ]
   },
   {
@@ -346,11 +346,11 @@
     "\n",
     "You can now test the fine-tuned model in two ways. First, you can visit the Playground and use the Models drop-down to select your newly fine-tuned model from the options listed. The other option is to use the \"Playground\" option shown in the Fine-tuning panel (see screenshot above) which launches the following _comparitive_ view which shows the foundation and fine-tuned model versions side-by-side for quick evaluation.\n",
     "\n",
-    "![Fine-tuning job status](./img/fine-tuned-playground-compare.png)\n",
+    "![Fine-tuning job status](./img/fine-tuned-playground-compare.png?WT.mc_id=academic-105485-koreyst)\n",
     "\n",
     "Simply fill in the system context used in your training data and provide your test question. You will notice that both sides are updated with the identical context and question. Run the comparison and you will see the difference in outputs between them. _Note how the fine-tuned model renders the response in the format you provided in your examples while the foundation model simply follows the system prompt_.\n",
     "\n",
-    "![Fine-tuning job status](./img/fine-tuned-playground-launch.png)\n",
+    "![Fine-tuning job status](./img/fine-tuned-playground-launch.png?WT.mc_id=academic-105485-koreyst)\n",
     "\n",
     "You will notice that that the comparison also provides the token counts for each model, and the time taken for the inference. **This specific example is a simplistic one meant to show the process but not actually reflecting a real world dataset or scenario**. You may notice that both samples show the same number of tokens (system context and user prompt are identical) with the fine-tuned model taking more time for inference (custom model).\n",
     "\n",