Real-Time IoT Perception Systems Based on Edge-Cloud Collaboration with Large Foundation Models

[bf-yang]

Introduction or background of this discussion:
OSPP project: "Real-Time IoT Perception Systems Based on Edge-Cloud Collaboration with Large Foundation Models"

Contents of this discussion:
Project Output Requirements:

1. Develop a Real-Time Perception Application System Based on Large Foundation Models on KubeEdge-Sedna (i.e., an edge-cloud collaboration platform), which supports efficient generalization capabilities.
2. The performance of the system will be tested on actual edge platforms (optional).

Project Technical Requirements:

1. Proficient in Python and have a basic understanding of edge platforms such as the NVIDIA Jetson series.
2. Familiar with at least one artificial intelligence framework and capable of developing algorithms and deploying them to edge platforms.

Project Description:
Real-time perception systems are an essential component of intelligent Internet of Things (IoT) devices such as industrial robots and household mobile devices. When performing basic perception tasks such as object detection, the limited resources of edge platforms pose great challenges to the accuracy and adaptiveness of models. For example, in the case of object detection, new environments bring new object classes and states, and the small models that edge platforms can only recognize limited-domain information. Currently, large foundation models represented by CLIP and GPT are widely recognized for their superior generalization ability. It is an important research direction to enable small models on edge platforms to achieve efficient and real-time IoT perception applications through the framework of edge-cloud collaboration with foundation models.

[MooreZheng]

Related issue and PR: #76 #80 #84