Skip to content

Commit

Permalink
adding ai builder content
Browse files Browse the repository at this point in the history
  • Loading branch information
mwasham committed Dec 9, 2024
0 parents commit cb1c54b
Show file tree
Hide file tree
Showing 6 changed files with 527 additions and 0 deletions.
133 changes: 133 additions & 0 deletions Exercise01.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# AI Builder Lab

AI Builder is a Microsoft platform feature that provides a user-friendly interface for building, training, and deploying custom artificial intelligence (AI) models without the need for extensive coding or data science expertise. Integrated with Microsoft Power Platform and Dynamics 365, it empowers users to automate tasks and gain insights from data through AI-driven solutions. Users can leverage pre-built AI models or create their own using the intuitive interface. Common applications include form processing, object detection, predictive modeling, and OpenAI. Overall, AI Builder democratizes AI, making it accessible to a broader audience within organizations.


## Exercise 1: Working with Documnents


## Overview

In this exercise, you will download sample data that will be used throughout the rest of the lab. From there, you will use a pre-created AI model to demonstrate how to programmatically parse an invoice in PDF Format.





### Task #1 - Verify your Microsoft Power Platform trial tenant

1. Verify that you have your **Microsoft 365 credentials** from the Authorized Lab Host available.

2. In a new browser tab, navigate to `https://make.powerapps.com`

3. Enter the `email address` provided by the Authorized Lab Host.

4. Select **Sign in**.

5. Enter the `password` provided by the Authorized Lab Host.

6. Optionally, select **Yes** to stay signed in.

7. If prompted, enter `0123456789` for **Phone number** and select **Submit**.

8. **Refresh** the tab and verify the **Dev One** environment is selected under **Environment** in the top right.

## Task 2: Download sample data

Throughout the lab, you will need to upload various files to test out AI Builders capabilities.

- Open an new tab and download the sample data file by typing this short url in the browser (inside the VM) [https://tinyurl.com/3zzcr5k5](https://opsgilitylabs.blob.core.windows.net/public/aibuilder/AIBuilderSampleData.zip) and unzip it's contents into the new folder.

## Task 3: Using an AI Model and Flow to Parse invoices

- Choose **AI Hub** from the navigation. If it is not visible you might have to click the **... More** navigation to add it to the menu.

![aihub](images/aihub.png)

- In the top navigation click on **Start free trial** for a 30 trial of AI Builder.

![aibuilderfree](images/aibuilderfree.png)

- Click on the large **AI Models** button

![aimodelsbtn](images/aimodelsbtn.png)

- In the main portion of the screen click on the **Documents** tab to filter the results.

![](images/invoices.png)

- Finally, select **Extract information from Invoices**

![](images/extract-from-invoices.png)


- In the dialog, click on **Use prebuilt model** and choose **Use in a flow** option from the dropdown. This means we will build a re-usable Power Automate Flow to create a re-usable Invoice Flow.

![](images/use-prebuilt-model.png)

![](images/useinflow1.png)

- There are several connections required for this demonstration to work. If you have never used that connection before the screen will resemble the following:

![](images/signing.png)

- Click **Sign in** for each connection. When you click **Sign in** you will see a quick dialog and when you have **Signed in** to all three connectors it will resemble the following with **Green** circled checks:

![](images/green.png)

- After that is complete you will need to click the **Continue** button

![](images/continue.png)

- This is using a pre-built model so there are no changes required to the **flow** presented

- Click on **Save** in the upper-right hand corner. Give it few seconds to complete.

![](images/save1.png)\

- Then click on the **Test** button at the top right of your screen.

![](images/test.png)

- You might also get another **Sign in** prompt. Click **Continue**

![](images/signin2.png)


- Click the **Import** button.

- Open the **AIBuilderLabFiles** folder that has the downloaded sample data. Then open the **DocumentProcessing_Invoices_Adatum** folder and select the **Test** folder. There is only a single invoice in there and it is a pdf file called **Adatum 6.pdf** file. Select it for the import.

- Then click the **Run flow** button at the bottom

![](images/adatum6.png)

- Flow is running. Then click **Done**

![](images/runsuccess.png)

- Since there were no changes made the **final** step in the **Flow** sends an email but you should see the following steps with **green** checks next to each step:

![](images/green1.png)

- Click the step called **Extract information from invoices.** There are two sections available in the step. **Inputs** and **Outputs**. The **Input** section will show a binary section of text that represents the submitted pdf file. In the **Outputs** section you can see all the extracted fields in JSON format (used by developers).

![](images/invoiceprocessed.png)

- If you want to see the email click on the **9 square** in the upper right hand corner. Then choose **Outlook** but choose **Open in new tab**

![](images/newtab.png)

- In **Outlook** open the item title **Invoice processed**



- Browse all the mapped fields in the email

![](images/invoiceemail.png)



## Summary

In this exercise, you created a flow that leveraged a pre-created AI Builder model to parse a PDF invoice and emailed the results all without writing any code.
64 changes: 64 additions & 0 deletions Exercise02.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
## Exercise 2

- Using the same in-private browser instance, navigate to the [Power Automate site](https://make.powerautomate.com/) if you are not already there.

- In the top right of the screen, ensure the Environment is set to **Dev One**.

- Choose **AI Hub** from the navigation. If it is not visible you might have to click the **... More** navigation to add it to the menu.

![aihub](images/aihub.png)

- Click on the large **AI Models** button

![aimodelsbtn](images/aimodelsbtn.png)

- In the main portion of the screen click on the **Documents** tab to filter the results.

![](images/invoices.png)

- In the main portion of the screen click on the **Documents** tab to filter the results. Finally, select **Extract all the text in photos and PDF Documents(OCR)**

![](images/textfromphotos.png)

- In the dialog, click on **Use prebuilt model** and choose **Use in a flow** option from the dropdown. This means we will build a re-usable Power Automate Flow to create a re-usable Flow to Extract all the text in photos and PDF documents (OCR).

- Very similiar to the previous exercise validate that you see green checks next to all the connections. Then click **Continue**

- Click on **Save** in the upper-right hand corner. Give it few seconds to complete.

![](images/save1.png)\

- Then click on the **Test** button

![](images/test.png)

- You might also get another **Sign in** prompt. Click **Continue**

![](images/signin2.png)


- Click the **Import** button.

- Open the **AIBuilderLabFiles** folder that has the downloaded sample data. Then open the **TextRecognizer** folder. Select the **TextReco-SamplePrinted.png** invoice for the import.

- Then click the **Run flow** button at the bottom

![](images/adatum6.png)

- Flow is running. Then click **Done**

![](images/runsuccess.png)

- Next visit the Outlook web email again and look for the **Text Recognition results** email.

![](images/textrecog.png)

- Review the results

![](images/reviewtextrecog.png)



## Summary

In this exercise, you created a flow that leveraged a pre-created AI Builder model to parse the text out of an image and emailed the results all without writing any code.
124 changes: 124 additions & 0 deletions Exercise03.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
## Exercise 3 - Extract information from receipts


- Using the same in-private browser instance, navigate to the [Power Apps site](https://make.powerapps.com/) site.

- In the top right of the screen, ensure the Environment is set to **Dev One**.

- Choose **AI Hub** from the navigation. If it is not visible you might have to click the **... More** navigation to add it to the menu.

![aihub](images/aihub.png)

- Click on the large **AI Models** button

![aimodelsbtn](images/aimodelsbtn.png)

- In the main portion of the screen click on the **Documents** tab to filter the results.

- Finally, select **Extract information from receipts**

![](images/textfromreceipts.png)

- In the dialog, click on **Use prebuilt model** and choose **Use in a flow** option from the dropdown. This means we will build a re-usable Power Automate Flow to create a re-usable Flow to Extract all the text in photos and PDF documents (OCR).

- Select the **Track expenses by scanning receipts to your OneDrive for Business** template that is already pre-created.

![](images/expenses.png)

- Very similiar to the previous exercise validate that you see green checks next to all the connections. Then click **Continue**

![](images/continue2.png)

> Note: If you get an error on the OneDrive for Business like the following graphic. The issue is because OneDrive hasnt been provisioned yet.
![onedriveerror](images/onedriveerror.png)

> Note: Then go to the Nine square button in the upper left hand corner. Right click on **OneDrive** and choose **Open in new tab**. Wait a few moments and you should see **Your OneDrive is ready**
![openinnewtab](images/openinnewtab.png)

> Note: Click on the three dots to attempt a re-connect again. All checks should be green now.
![reconnect](images/reconnect.png)

- Before configuring the **Flow** click on the nine square in the upper-right hand corner. Then select the three dots next to **OneDrive** and choose **Open in new tab**

![](images/onedrivetab.png)

- In OneDrive click on the **Add new** button and choose **Files upload**

- Upload the following file **Receipts.xlsx** from the Lab Downloads file in the **AIBuilderLabFiles\ReceiptsProcessing_Contoso** folder.

- Return to the **Flow** tab in the browser

- There are two steps that need additional configuration. Select the first step called **When a file is created**. When selected the properties panel will open. Click on the **folder** icon and choose **Root** to select the top level folder in **OneDrive**.

![](images/rootselected.png)

- Now select the fourth, and last step, called **Add a row into a table**. **Note:** this does require the file to uploaded from the previous step.

![](images/add-a-row.png)


- In the properties panel choose the **Document Library** and select **OneDrive**

![](images/onedrive2.png)

- Next click on the folder icon and select the **Receipts.xlsx** file previously uploaded.

- In the **Table** dropdown select **Table1** which is defined in the Excel spreadsheet that was uploaded to OneDrive.

![](images/ttable1.png)


- The view after configured.

![](images/add-table-final.png)


- In the **Advanced parameters** section you should see some pre-populated values. Click on the **Show all** link. This will show all the potential values available.

![](images/showall.png)

- From the **DateTime Format** dropdown choose **ISO 8601**

![](images/iso.png)

- Add additional parameters to fill in the Excel Spreadhsheet you can use the following animation to assist.

![](images/parmsmapping.gif)


- Click on **Save** in the upper right-hand corner

![](images/save1.png)

- Remove the following unnecessary options by clicking the X button. Then click **Save** again.

![](images/remove-fields.png)


- Then click on **Test** and choose the radio button for **Manually** and finally click the **Test** button. This will start the flow to monitor for uploaded files to OneDrive.

![](images/test3.png)


- Switch back to the OneDrive tab and upload one of the sample receipts from the C:\LabFiles\ReceiptsProcessing_Contoso folder.

- This shows the completed **Flow**

![](images/receiptsflow.png)

- Now switch back to the **OneDrive** tab and open the **Receipts.xlsx** file

- Depending on the parameters added and file chosen you might see different values

![](images/receiptout.png)

> The values were parsed from the uploaded reciept and stored in the spreadsheet.


## Summary

In this exercise, you created a flow that leveraged a pre-created AI Builder model to parse data out of uploaded reciepts and saved the results in a spreadsheet all without writing any code.
Loading

0 comments on commit cb1c54b

Please sign in to comment.