Skip to content

Commit

Permalink
Update: update AI workshop student wiki
Browse files Browse the repository at this point in the history
  • Loading branch information
@MatthewJeffson
MatthewJeffson committed Sep 26, 2024
1 parent 819979d commit 5093cb1
Show file tree
Hide file tree
Showing 3 changed files with 222 additions and 14 deletions.
33 changes: 23 additions & 10 deletions docs/Topics/TinyML/TinyML_Workshop/TinyML_workshop_course.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -72,43 +72,56 @@ To complete the workshop, we need go through the instructions below, and each pa

## Getting Started

### Step 1: Familiarize the tools that we are using.
### Step 1: The AI "Blink": Quickly Using AI

<div class="all_container">
<div class="getting_started">
<div class="start_card_wrapper">
<a href= "/sscma" class="getting_started_label2">1 SenseCraft AI Platform</a>
<br/>Use pre-made tinyML models and experiment quickly.
<a href= "/sscma" class="getting_started_label2">1.1 SenseCraft AI Platform Introduction</a>
<br/>Familiarize the tools that we are using.
</div>
</div>
<div class="getting_started">
<div class="start_card_wrapper">
<a href= "/sscma/#the-ai-blink-quickly-using-ai" class="getting_started_label2">1.2 The AI "Blink" </a>
<br/>Learn how to easily deploy public AI model libraries on edge devices, XIAO ESP32S3 Sense.
</div>
</div>
</div>

### Step 2: Experiment tinyML models with pre-built arduino library
### Step 2: Train and Deploy Your Own AI Model

<div class="all_container">
<div class="getting_started">
<div class="start_card_wrapper">
<a href= "/sscma" class="getting_started_label2">2 The AI "Blink" </a>
<br/>Learn how to easily deploy public AI model libraries on edge devices.
<a href= "/train_and_deploy_model" class="getting_started_label2">2. Build Your own project </a>
<br/>Learn how to easily train and deploy your own models.
</div>
</div>
</div>

### Step 3: Make your own tinyML project by collecting data and training your own.
### Step 3: Federate(Optional)

<div class="all_container">
<div class="getting_started">
<div class="start_card_wrapper">
<a href= "/train_and_deploy_model" class="getting_started_label2">3 Build Your own project </a>
<br/>Learn how to easily train and deploy your own models.
<a href= "/train_and_deploy_model/#optional-federate-transmit-data-wide-away" class="getting_started_label2">3. Transmit data wide away </a>
<br/>Learn how to federate your models wide away in SenseCraft.
</div>
</div>
</div>

Please go one by one to the above topic and complete the ToDo to go to the next level. All the best and happy making 🙌.

## ✨ Collaborations ✨

This course is open source and we welcome more collaboration! You can refer to [here](https://wiki.seeedstudio.com/Contribution-Guide/) to see the Contribution(Pull Request) Guide.

:::info
These files are under the relative path of `docs/Topics/TinyML/TinyML_Workshop`.
:::

## Edge Impulse
## (Other Method)Edge Impulse

<div class="button_tech_support_container">
<a href="https://edgeimpulse.com/" class="button_edgeimpulse"></a>
Expand Down
102 changes: 102 additions & 0 deletions docs/Topics/TinyML/TinyML_Workshop/TrainModel.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -100,12 +100,114 @@ If you have more time, you can try using [the `Output` operation you've learned
:::



# ToDo
- [ ] Train and deploy models using the SenseCraft AI platform.
- [ ] Setup a Trigger and **Control LED** for your trained models with SenseCraft AI Platform.

## (Optional) Federate: Transmit data wide away

In this step, we are trying to transmit our data from XIAO ESP32S3 Sense to a remote device, using Wi-Fi and MQTT, helping us see its deployment remotely.

### Step 1. Setup MQTT on a device and test

As example, we are using NVIDIA Jetson reComputer J4012 for it. It supports MQTT broker installation and the most important thing is that it offers 100 TOPS AI Power, for us applying LLM locally.

The SenseCraft AI Platform is supporting Wi-Fi and MQTT connection.

<iframe width={800} height={480} src="https://www.youtube.com/embed/-KAyUHzRxHc" title="Unboxing & Plug in reComputer J4012 - Powered by NVIDIA Jetson Orin NX" frameBorder={0} allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerPolicy="strict-origin-when-cross-origin" allowFullScreen />
First we need to install the MQTT broker(Mosquitto) and then try to setup the MQTT server.
```
sudo apt-get update
sudo apt-get install mosquitto
```
and get the Mosquitto installation done on the reComputer(Linux).
Then run the commend:
```
sudo service mosquitto start
```
to start it.
Later we can run this commend:
```
sudo service mosquitto status
```
to see whether it is been activited:
![image](https://fabacademy.org/2024/labs/chaihuo/students/matthew-yu/assets/images/mqtt_docusaurus_xiao_1-3919de85499db74b41cf3057bcdfe6bd.png)
:::info
Testing:
For creating/subcribing a topic:
```
mosquitto_sub -h localhost -t "LED"
```
For sending/publishing some data:
```
mosquitto_pub -h localhost -t "LED" -m "1"
mosquitto_pub -h localhost -t "LED" -m "test"
```
Getting the results and it seems all good:
![image](https://fabacademy.org/2024/labs/chaihuo/students/matthew-yu/assets/images/mqtt_docusaurus_xiao_3-281bf87c08ecdb601595625229a7e1df.png)
:::
And the `localhost` is `192.168.66.184`(as reComputer):
![image](https://fabacademy.org/2024/labs/chaihuo/students/matthew-yu/assets/images/mqtt_docusaurus_xiao_2-8202adc158ca9aa540a264c288c431ed.jpg)
### Step 3. Configure XIAO ESP32S3 Sense on SenseCraft AI Platform
On the SenseCraft AI Platform, you can refer to the "Configuration" page:
<div style={{textAlign:'center'}}><img src="https://files.seeedstudio.com/wiki/tinyml-topic/xiao_mqtt_1.png" style={{width:800, height:'auto'}}/></div>
:::info
- SSID: (Wi-Fi name same as your MQTT device)
- Password: (Wi-Fi passowrd as your MQTT device)
- Encryption: AUTO
- MQTT: Yes
- Host: (IP address from your MQTT device)
- Port: 1883
In this exaple, the MQTT device is the reComputer like above.
:::
### Step 3. Receive data from XIAO ESP32S3 Sense and Display
In the receive part, you can install a client with command:
```
pip install python-sscma
```
This is an integrated client for the [sscma_micro](https://github.com/Seeed-Studio/sscma_micro), which is a microcontroller at server for the [SSCMA](https://github.com/Seeed-Studio/SSCMA) models.
And then receive the data using:
```
sscma.cli client --broker mqtt.broker.com --device device_id
```
:::info
In this case, `mqtt.broker.com` is 192.168.66.184, `device_id` is from XIAO ESP32S3 Sense on SenseCraft AI Platform.
<div style={{textAlign:'center'}}><img src="https://files.seeedstudio.com/wiki/tinyml-topic/xiao_mqtt_2.png" style={{width:300, height:'auto'}}/></div>
:::
### Step 4. (comming soon)Federate multiple XIAOs on one page
### Step 5. (comming soon)Enable LLM to automatically watch images from XIAO
101 changes: 97 additions & 4 deletions docs/Topics/TinyML/TinyML_Workshop/sscma.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,113 @@ keywords:
image: https://files.seeedstudio.com/wiki/seeed_logo/logo_2023.png
slug: /sscma
last_update:
date: 08/25/2024
author: Salman
date: 08/26/2024
author: Matthew
---

## SenseCraft AI Platform
## Familiarize the tools that we are using

Seeed Studio [SenseCraft AI Platform](https://sensecraft.seeed.cc/ai/#/model) is a browser-based AI Solution.
### SenseCraft AI Platform

Seeed Studio [SenseCraft AI Platform](https://sensecraft.seeed.cc/ai/#/model) is a browser-based AI Solution.

It empowers users to effortlessly deploy a vast library of publicly available AI models onto their edge devices provides a seamless and user-friendly experience, allowing you to deploy public AI models directly onto your edge devices with **just a few clicks**.

:::info
The core of it is an open source project and we have share it on the [GitHub](https://github.com/Seeed-Studio/ModelAssistant) and offer the [development method](/ModelAssistant_Introduce_Overview) as well.
:::

#### User Account

[SenseCraft AI](https://sensecraft.seeed.cc/ai/#/model) is an open platform that allows users to browse all public AI models and Home pages without logging in. You need to sign up and sign in only when you need to deploy a model, or share your own model.

[SenseCraft AI](https://sensecraft.seeed.cc/ai/#/model) and [SenseCraft Data Platform](https://sensecap.seeed.cc/portal/#/login) ( original SenseCAP Cloud Platform) are both software services provided by seeed studio for users, users only need to sign up for an account on any one of the platforms, and then they can use the same account to sign in on both platforms.

#### Sign up

1. Enter your name and valid email,and then click "get capcha"<br />


![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/1.png)

2. Get the verification code from your email and enter it on sign up page

:::note
verification code is valid for 10min, please complete the registration within 10 minutes
:::

![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/2.png)

3. Enter your password and other user information to complete your registration. <br />

![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/3.png)

#### Sign in

Sign in with your registered email account

![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/4.png)

More about SenseCraft AI platform you can refer to [here](https://wiki.seeedstudio.com/sensecraft_ai/).

## AI Models on SenseCraft AI Platform

### Model list

Public AI models show all published AI models；Custom AI models show all your own uploaded AI models. The model is displayed on the list as follows

- Model inference example image
- Model Name
- Model describe
- This model task type like detection,classification,segmentation,pose, generation
- Supported Device:Device that support deployment of this model
- The username that published this model

**Search model**<br/>
Provides search feature based on various types of key information in the model like model name, model task, surpported device,model publisher

**Favorite model**<br/>
Users can favorite public models

![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/7.png)

More information about AI Models in SenseCraft AI platform you can refer to [here](https://wiki.seeedstudio.com/sensecraft_ai/#ai-models).

## Device Workspace

The SenseCraft AI platform provides an AI model zoo for users to share and download AI models, and also has a device workspace where users can view the inference results of their devices in real time and manage their devices to improve the convenience and flexibility of device management.

### XIAO ESP32S3 Workspace

![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/image4.png)

You can learn more through the [XIAO ESP32S3 Workspace documentation](https://wiki.seeedstudio.com/XIAO_ESP32S3_Workspace/).

### Grove-Vision AI v2 Workspace

![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/image10.png)

You can learn more through the [Grove-Vision AI v2 Workspace documentation](https://wiki.seeedstudio.com/Grove_Vision_AI_v2_Workspace/).

### Nvidia Jeson Workspace

![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/35.png)

You can learn more through the [Nvidia Jeson Workspace documentation](https://wiki.seeedstudio.com/Nvidia_Jeson_Workspace/).

#### SenseCraft AI-Jetson

SenseCraft AI-Jetson is a development toolkit and platform designed for NVIDIA Jetson Edge AI devices. Simply run the "Quickstart Script" and you will be presented with an interactive user interface to view an example application with a pre-loaded video and a pre-loaded AI model. If you want to add your own USB camera, IP camera you can simply do it in a few clicks!

In addition to the various built-in AI models we offer out-of-the-box, you can access to a large number of public models on SenseCraft AI platform and you will be able to download and deploy AI models for specific scenarios and create personalized AI solutions based on your needs. SenseCraft AI is your intelligent decision-making partner for vision ai, providing you with simple, flexible and efficient inferencing and solution building capabilities.

![](https://files.seeedstudio.com/wiki/SenseCraft_AI/img/video.gif)


You can learn more through the [SenseCraft AI-Jetson documentation](https://wiki.seeedstudio.com/SenseCraft_AI_Jetson/).

## The AI "Blink": Quickly Using AI

### 1. Get started with Face Recognition

We first go to the [SenseCraft AI Deployment Website](https://sensecraft.seeed.cc/ai/#/device/local?time=1724577953974) then simply connect the XIAO ESP32S3 Sense to your PC via a data cable to instantly display face recognition.
Expand Down

0 comments on commit 5093cb1

Please sign in to comment.