diff --git a/contents/ai_for_good/ai_for_good.qmd b/contents/ai_for_good/ai_for_good.qmd index d94e93ff..8197a6f5 100644 --- a/contents/ai_for_good/ai_for_good.qmd +++ b/contents/ai_for_good/ai_for_good.qmd @@ -38,7 +38,7 @@ What is special about the SDGs is that they are a collection of interlinked obje A recent study [@vinuesa2020role] highlights the influence of AI on all aspects of sustainable development, particularly on the 17 Sustainable Development Goals (SDGs) and 169 targets internationally defined in the 2030 Agenda for Sustainable Development. The study shows that AI can act as an enabler for 134 targets through technological improvements, but it also highlights the challenges of AI on some targets. The study shows that AI can benefit 67 targets when considering AI and societal outcomes. Still, it also warns about the issues related to the implementation of AI in countries with different cultural values and wealth. -![United Nations Sustainable Development Goals (SDG). Source: [United Nations](https://sdgs.un.org/goals).](https://www.un.org/sustainabledevelopment/wp-content/uploads/2015/12/english_SDG_17goals_poster_all_languages_with_UN_emblem_1.png){#fig-sdg} +![United Nations Sustainable Development Goals (SDG). Source: [United Nations](https://sdgs.un.org/goals).](https://www.un.org/sustainabledevelopment/wp-content/uploads/2015/12/english_SDG_17goals_poster_all_languages_with_UN_emblem_1.png){#fig-fig-sdg} In our book's context, TinyML could help advance at least some of these SDG goals. @@ -68,7 +68,7 @@ Other sensors, such as GPS units and accelerometers, can track microclimate cond Widespread TinyML applications can help digitize smallholder farms to increase productivity, incomes, and resilience. The low cost of hardware and minimal connectivity requirements make solutions accessible. Projects across the developing world have shown the benefits: -* Microsoft's [FarmBeats](https://www.microsoft.com/en-us/research/project/farmbeats-iot-agriculture/) project is an end-to-end approach to enable data-driven farming by using low-cost sensors, drones, and vision and machine learning algorithms. The project seeks to solve the problem of limited adoption of technology in farming due to the need for more power and internet connectivity in farms and the farmers' limited technology savviness. The project strives to increase farm productivity and reduce costs by coupling data with farmers' knowledge and intuition about their farms. The project has successfully enabled actionable insights from data by building artificial intelligence (AI) or machine learning (ML) models based on fused data sets. @farmbeats illustrates the inner workings of Microsoft's FarmBeats. +* Microsoft's [FarmBeats](https://www.microsoft.com/en-us/research/project/farmbeats-iot-agriculture/) project is an end-to-end approach to enable data-driven farming by using low-cost sensors, drones, and vision and machine learning algorithms. The project seeks to solve the problem of limited adoption of technology in farming due to the need for more power and internet connectivity in farms and the farmers' limited technology savviness. The project strives to increase farm productivity and reduce costs by coupling data with farmers' knowledge and intuition about their farms. The project has successfully enabled actionable insights from data by building artificial intelligence (AI) or machine learning (ML) models based on fused data sets. @fig-farmbeats illustrates the inner workings of Microsoft's FarmBeats. * In Sub-Saharan Africa, off-the-shelf cameras and edge AI have cut cassava disease losses from 40% to 5%, protecting a staple crop [@ramcharan2017deep]. @@ -76,7 +76,7 @@ Widespread TinyML applications can help digitize smallholder farms to increase p With greater investment and integration into rural advisory services, TinyML could transform small-scale agriculture and improve farmers' livelihoods worldwide. The technology effectively brings the benefits of precision agriculture to disconnected regions most in need. -![Microsoft Farmbeats.](contents/ai_for_good/images/png/msfarmbeats.png){#farmbeats} +![Microsoft Farmbeats enables data-driven decisions to improve agricultural yield, lower overall costs, and reduce the environmental impact of agricultural production. Source: [MOLD](https://thisismold.com/space/farm-systems/does-farmbeats-really-march-to-a-different-tune)](images/png/msfarmbeats.png){#fig-farmbeats} :::{#exr-agri .callout-caution collapse="true"} diff --git a/contents/benchmarking/benchmarking.qmd b/contents/benchmarking/benchmarking.qmd index 25d55825..4a22e161 100644 --- a/contents/benchmarking/benchmarking.qmd +++ b/contents/benchmarking/benchmarking.qmd @@ -242,8 +242,6 @@ Benchmarking the training phase provides insights into how different model archi On the other hand, benchmarking inference evaluates model performance in real-world conditions after deployment. Key metrics include latency, throughput, memory footprint, and power consumption. This type of benchmarking determines if a model meets the requirements of its target application regarding response time and device constraints. However, we will discuss these broadly to ensure a general understanding. -![Deep learning: training vs. inference.](contents/benchmarking/images/png/trainingvsinference.png){#training-vs-inference} - ### Training Benchmarks Training represents the phase where the system processes and ingests raw data to adjust and refine its parameters. Therefore, it is an algorithmic activity and involves system-level considerations, including data pipelines, storage, computing resources, and orchestration mechanisms. The goal is to ensure that the ML system can efficiently learn from data, optimizing both the model's performance and the system's resource utilization. @@ -750,9 +748,9 @@ There are several benefits to adopting a data-centric approach to AI development Additionally, a data-centric approach can often lead to simpler models that are easier to interpret and maintain. This is because the emphasis is on the data rather than the model architecture, meaning simpler models can achieve high performance when trained on high-quality data. -The shift towards data-centric AI represents a significant paradigm shift. By prioritizing the quality of the input data, this approach tries to model performance and generalization capabilities, ultimately leading to more robust and reliable AI systems. As we continue to advance in our understanding and application of AI, the data-centric approach is likely to play an important role in shaping the future of this field. +The shift towards data-centric AI represents a significant paradigm shift. By prioritizing the quality of the input data, this approach tries to model performance and generalization capabilities, ultimately leading to more robust and reliable AI systems. @fig-data-vs-model illustrates this difference. As we continue to advance in our understanding and application of AI, the data-centric approach is likely to play an important role in shaping the future of this field. -![Model-centric ML development vs. Data-centric ML development.](contents/benchmarking/images/png/datavsmodelai.png){#data-vs-model} +![Model-centric vs. Data-centric ML development. Source: [NVIDIA](https://blogs.nvidia.com/blog/difference-deep-learning-training-inference-ai/)](images/png/datavsmodelai.png){#fig-data-vs-model} ### Benchmarking Data diff --git a/contents/data_engineering/data_engineering.qmd b/contents/data_engineering/data_engineering.qmd index f561792d..71ae6336 100644 --- a/contents/data_engineering/data_engineering.qmd +++ b/contents/data_engineering/data_engineering.qmd @@ -247,9 +247,9 @@ Data sourcing and data storage go hand in hand, and data must be stored in a for : Comparative overview of the database, data warehouse, and data lake. {#tbl-storage .striped .hover} -The stored data is often accompanied by metadata, defined as 'data about data .'It provides detailed contextual information about the data, such as means of data creation, time of creation, attached data use license, etc. For example, [Hugging Face](https://huggingface.co/) has [Dataset Cards](https://huggingface.co/docs/hub/datasets-cards). To promote responsible data use, dataset creators should disclose potential biases through the dataset cards. These cards can educate users about a dataset's contents and limitations. The cards also give vital context on appropriate dataset usage by highlighting biases and other important details. Having this type of metadata can also allow fast retrieval if structured properly. Once the model is developed and deployed to edge devices, the storage systems can continue to store incoming data, model updates, or analytical results. @data-collection showcases the pillars of data collection and their collection methods. +The stored data is often accompanied by metadata, defined as 'data about data.' It provides detailed contextual information about the data, such as means of data creation, time of creation, attached data use license, etc. For example, [Hugging Face](https://huggingface.co/) has [Dataset Cards](https://huggingface.co/docs/hub/datasets-cards). To promote responsible data use, dataset creators should disclose potential biases through the dataset cards. These cards can educate users about a dataset's contents and limitations. The cards also give vital context on appropriate dataset usage by highlighting biases and other important details. Having this type of metadata can also allow fast retrieval if structured properly. Once the model is developed and deployed to edge devices, the storage systems can continue to store incoming data, model updates, or analytical results. @fig-data-collection showcases the pillars of data collection and their collection methods. -![Pillars of data collection.](contents/data_engineering/images/png/datacollection.png){#data-collection} +![Pillars of data collection. Source: [Alexsoft](https://www.altexsoft.com/blog/data-collection-machine-learning/)](images/png/datacollection.png){#fig-data-collection} **Data Governance:** With a large amount of data storage, it is also imperative to have policies and practices (i.e., data governance) that help manage data during its life cycle, from acquisition to disposal. Data governance outlines how data is managed and includes making key decisions about data access and control. @fig-governance illustrates the different domains involved in data governance. It involves exercising authority and making decisions concerning data to uphold its quality, ensure compliance, maintain security, and derive value. Data governance is operationalized by developing policies, incentives, and penalties, cultivating a culture that perceives data as a valuable asset. Specific procedures and assigned authorities are implemented to safeguard data quality and monitor its utilization and related risks. diff --git a/contents/dl_primer/dl_primer.qmd b/contents/dl_primer/dl_primer.qmd index b64cd610..98c8ad1d 100644 --- a/contents/dl_primer/dl_primer.qmd +++ b/contents/dl_primer/dl_primer.qmd @@ -54,11 +54,11 @@ Organizations worldwide recognize deep learning's transformative potential and i ### Applications of Deep Learning -Deep learning is extensively used across numerous industries today, and its transformative impact on society is evident. In finance, it powers stock market prediction, risk assessment, and fraud detection. For instance, deep learning algorithms can predict stock market trends, guide investment strategies, and improve financial decisions. In marketing, it drives customer segmentation, personalization, and content optimization. Deep learning analyzes consumer behavior and preferences to enable highly targeted advertising and personalized content delivery. In manufacturing, deep learning streamlines production processes and enhances quality control by continuously analyzing large volumes of data. This allows companies to boost productivity and minimize waste, leading to the production of higher quality goods at lower costs. In healthcare, machine learning aids in diagnosis, treatment planning, and patient monitoring. Similarly, deep learning can make medical predictions that improve patient diagnosis and save lives. The benefits are clear: machine learning predicts with greater accuracy than humans and does so much more quickly.@deeplearning further illustrates some applications of deep learning. +Deep learning is extensively used across numerous industries today, and its transformative impact on society is evident. In finance, it powers stock market prediction, risk assessment, and fraud detection. For instance, deep learning algorithms can predict stock market trends, guide investment strategies, and improve financial decisions. In marketing, it drives customer segmentation, personalization, and content optimization. Deep learning analyzes consumer behavior and preferences to enable highly targeted advertising and personalized content delivery. In manufacturing, deep learning streamlines production processes and enhances quality control by continuously analyzing large volumes of data. This allows companies to boost productivity and minimize waste, leading to the production of higher quality goods at lower costs. In healthcare, machine learning aids in diagnosis, treatment planning, and patient monitoring. Similarly, deep learning can make medical predictions that improve patient diagnosis and save lives. The benefits are clear: machine learning predicts with greater accuracy than humans and does so much more quickly. @fig-deeplearning further illustrates some applications of deep learning. Deep learning enhances everyday products, such as strengthening Netflix's recommender systems to provide users with more [personalized recommendations](https://dl.acm.org/doi/abs/10.1145/3543873.3587675). At Google, deep learning models have driven significant improvements in [Google Translate](https://research.google/blog/recent-advances-in-google-translate/), enabling it to handle over [100 languages](https://cloud.google.com/translate/docs/languages). Autonomous vehicles from companies like Waymo, Cruise, and Motional have become a reality through the use of deep learning in their [perception system](https://motional.com/news/technically-speaking-improving-av-perception-through-transformative-machine-learning). Additionally, Amazon employs deep learning at the edge in their Alexa devices to perform [keyword spotting](https://towardsdatascience.com/how-amazon-alexa-works-your-guide-to-natural-language-processing-ai-7506004709d3). -![Deep learning applications, benefits and implementations.](contents/dl_primer/images/png/deeplearning.png){#deeplearning} +![Deep learning applications, benefits and implementations. Source: [Leeway Hertz](https://www.leewayhertz.com/what-is-deep-learning/)](images/png/deeplearning.png){#fig-deeplearning} ### Relevance to Embedded AI @@ -120,15 +120,15 @@ The training process involves several key steps, beginning with the forward pass #### Forward Pass -The forward pass is the initial phase where data moves through the network from the input to the output layer. At the start of training, the network's weights are randomly initialized, setting the initial conditions for learning. During the forward pass, each layer performs specific computations on the input data using these weights and biases, and the results are then passed to the subsequent layer. The final output of this phase is the network’s prediction. This prediction is compared to the actual target values present in the dataset to calculate the loss, which can be thought of as the difference between the predicted outputs and the target values. The loss quantifies the network’s performance at this stage, providing a crucial metric for the subsequent adjustment of weights during the backward pass. +The forward pass is the initial phase where data moves through the network from the input to the output layer. At the start of training, the network's weights are randomly initialized, setting the initial conditions for learning. During the forward pass, each layer performs specific computations on the input data using these weights and biases, and the results are then passed to the subsequent layer. The final output of this phase is the network's prediction. This prediction is compared to the actual target values present in the dataset to calculate the loss, which can be thought of as the difference between the predicted outputs and the target values. The loss quantifies the network's performance at this stage, providing a crucial metric for the subsequent adjustment of weights during the backward pass. @vid-nn below explains how neural networks work using handwritten digit recognition as an example application. It also touches on the math underlying neural nets. :::{#vid-nn .callout-important} -@forward-propagation explains the concept of forward pass using an illustration. +@fig-forward-propagation explains the concept of forward pass using an illustration. -![Neural networks - forward and backward propagation.](contents/dl_primer/images/png/forwardpropagation.png){#forward-propagation} +![Neural networks - forward and backward propagation. Source: [Linkedin](https://www.linkedin.com/pulse/lecture2-unveiling-theoretical-foundations-ai-machine-underdown-phd-oqsuc/)](images/png/forwardpropagation.png){#fig-forward-propagation} # Neural Networks @@ -233,7 +233,7 @@ These architectures serve specific purposes and excel in different domains, offe ### Traditional ML vs Deep Learning -Deep learning extends traditional machine learning by utilizing neural networks to discern patterns in data. In contrast, traditional machine learning relies on a set of established algorithms such as decision trees, k-nearest neighbors, and support vector machines, but does not involve neural networks. To briefly highlight the differences, @tbl-mlvsdl illustrates the contrasting characteristics between traditional ML and deep learning. @ml-vs-dl further explains the differences between Machine Learning and Deep Learning. +Deep learning extends traditional machine learning by utilizing neural networks to discern patterns in data. In contrast, traditional machine learning relies on a set of established algorithms such as decision trees, k-nearest neighbors, and support vector machines, but does not involve neural networks. To briefly highlight the differences, @tbl-mlvsdl illustrates the contrasting characteristics between traditional ML and deep learning. @fig-ml-dl further explains the differences between Machine Learning and Deep Learning. +-------------------------------+-----------------------------------------------------------+--------------------------------------------------------------+ | Aspect | Traditional ML | Deep Learning | @@ -251,7 +251,7 @@ Deep learning extends traditional machine learning by utilizing neural networks | Maintenance | Easier (simple to update and maintain) | Complex (requires more efforts in maintenance and updates) | +-------------------------------+-----------------------------------------------------------+--------------------------------------------------------------+ -![Comparing Machine Learning and Deep Learning.](contents/dl_primer/images/png/mlvsdl.png){#ml-vs-dl} +![Comparing Machine Learning and Deep Learning. Source: [Medium](https://aoyilmaz.medium.com/understanding-the-differences-between-deep-learning-and-machine-learning-eb41d64f1732)](images/png/mlvsdl.png){#fig-ml-dl} : Comparison of traditional machine learning and deep learning. {#tbl-mlvsdl .striped .hover} diff --git a/contents/efficient_ai/efficient_ai.qmd b/contents/efficient_ai/efficient_ai.qmd index 340db085..9ebd1b27 100644 --- a/contents/efficient_ai/efficient_ai.qmd +++ b/contents/efficient_ai/efficient_ai.qmd @@ -76,21 +76,20 @@ Model compression methods are essential for bringing deep learning models to dev ![Different forms of quantization.](images/jpg/quantization.jpeg){#fig-quantization} -**Knowledge Distillation:** Knowledge distillation involves training a smaller model (student) to replicate the behavior of a larger model (teacher). The idea is to transfer the knowledge from the cumbersome model to the lightweight one. Hence, the smaller model attains performance close to its larger counterpart but with significantly fewer parameters. @knowledge-dist demonstrates the tutor-student framework for knowledge distillation. We will explore knowledge distillation in more detail in the @sec-kd. +**Knowledge Distillation:** Knowledge distillation involves training a smaller model (student) to replicate the behavior of a larger model (teacher). The idea is to transfer the knowledge from the cumbersome model to the lightweight one. Hence, the smaller model attains performance close to its larger counterpart but with significantly fewer parameters. @fig-knowledge-dist demonstrates the tutor-student framework for knowledge distillation. We will explore knowledge distillation in more detail in the @sec-kd. + +![The tutor-student framework for knowledge distillation. Source: [Medium](URL:https://chukwubuikexo.medium.com/knowledge-distillation-approaches-in-machine-learning-5841a41a346a)](images/png/knowledgedistillation.png){#fig-knowledge-dist} -![The tutor-student framework for knowledge distillation.](contents/efficient_ai/images/png/knowledgedistillation.png){#knowledge-dist} ## Efficient Inference Hardware -In the [Training](../training/training.qmd) chapter, we discussed the process of training AI models. Now, from an efficiency standpoint, it’s important to note that training is a resource and time-intensive task, often requiring powerful hardware and taking anywhere from hours to weeks to complete. Inference, on the other hand, needs to be as fast as possible, especially in real-time applications. This is where efficient inference hardware comes into play. By optimizing the hardware specifically for inference tasks, we can achieve rapid response times and power-efficient operation, which is especially crucial for edge devices and embedded systems. +In the [Training](../training/training.qmd) chapter, we discussed the process of training AI models. Now, from an efficiency standpoint, it's important to note that training is a resource and time-intensive task, often requiring powerful hardware and taking anywhere from hours to weeks to complete. Inference, on the other hand, needs to be as fast as possible, especially in real-time applications. This is where efficient inference hardware comes into play. By optimizing the hardware specifically for inference tasks, we can achieve rapid response times and power-efficient operation, which is especially crucial for edge devices and embedded systems. **TPUs (Tensor Processing Units):** [TPUs](https://cloud.google.com/tpu) are custom-built ASICs (Application-Specific Integrated Circuits) by Google to accelerate machine learning workloads [@jouppi2017datacenter]. They are optimized for tensor operations, offering high throughput for low-precision arithmetic, and are designed specifically for neural network machine learning. TPUs significantly accelerate model training and inference compared to general-purpose GPU/CPUs. This boost means faster model training and real-time or near-real-time inference capabilities, crucial for applications like voice search and augmented reality. [Edge TPUs](https://cloud.google.com/edge-tpu) are a smaller, power-efficient version of Google's TPUs tailored for edge devices. They provide fast on-device ML inferencing for TensorFlow Lite models. Edge TPUs allow for low-latency, high-efficiency inference on edge devices like smartphones, IoT devices, and embedded systems. AI capabilities can be deployed in real-time applications without communicating with a central server, thus saving bandwidth and reducing latency. Consider the table in @fig-edge-tpu-perf. It shows the performance differences between running different models on CPUs versus a Coral USB accelerator. The Coral USB accelerator is an accessory by Google's Coral AI platform that lets developers connect Edge TPUs to Linux computers. Running inference on the Edge TPUs was 70 to 100 times faster than on CPUs. -![Accelerator vs CPU performance comparison across different hardware configurations. Desktop CPU: 64-bit Intel(R) Xeon(R) E5–1650 v4 @ 3.60GHz. Embedded CPU: Quad-core Cortex-A53 @ 1.5GHz, †Dev Board: Quad-core Cortex-A53 @ 1.5GHz + Edge TPU. Source: [TensorFlow Blog.](https://blog.tensorflow.org/2019/03/build-ai-that-works-offline-with-coral.html)](images/png/tflite_edge_tpu_perf.png){#fig-edge-tpu-perf} - - +![Accelerator vs CPU performance comparison across different hardware configurations. Desktop CPU: 64-bit Intel(R) Xeon(R) E5-1650 v4 @ 3.60GHz. Embedded CPU: Quad-core Cortex-A53 @ 1.5GHz, †Dev Board: Quad-core Cortex-A53 @ 1.5GHz + Edge TPU. Source: [TensorFlow Blog.](https://blog.tensorflow.org/2019/03/build-ai-that-works-offline-with-coral.html)](images/png/tflite_edge_tpu_perf.png){#fig-edge-tpu-perf} **NN (Neural Network) Accelerators:** Fixed-function neural network accelerators are hardware accelerators designed explicitly for neural network computations. They can be standalone chips or part of a larger system-on-chip (SoC) solution. By optimizing the hardware for the specific operations that neural networks require, such as matrix multiplications and convolutions, NN accelerators can achieve faster inference times and lower power consumption than general-purpose CPUs and GPUs. They are especially beneficial in TinyML devices with power or thermal constraints, such as smartwatches, micro-drones, or robotics. diff --git a/contents/frameworks/frameworks.qmd b/contents/frameworks/frameworks.qmd index e38d2b64..c369eb7d 100644 --- a/contents/frameworks/frameworks.qmd +++ b/contents/frameworks/frameworks.qmd @@ -74,9 +74,9 @@ Each generation of frameworks unlocked new capabilities that powered advancement * TensorFlow Graphics (2020) added 3D data structures to handle point clouds and meshes. -In recent years, the frameworks have converged. @fig-ml-framework shows that TensorFlow and PyTorch have become the overwhelmingly dominant ML frameworks, representing more than 95% of ML frameworks used in research and production. @tensorflow-pytorch draws a contrast between the attributes of TensorFlow and PyTorch. Keras was integrated into TensorFlow in 2019; Preferred Networks transitioned Chainer to PyTorch in 2019; and Microsoft stopped actively developing CNTK in 2022 to support PyTorch on Windows. +In recent years, the frameworks have converged. @fig-ml-framework shows that TensorFlow and PyTorch have become the overwhelmingly dominant ML frameworks, representing more than 95% of ML frameworks used in research and production. @fig-tensorflow-pytorch draws a contrast between the attributes of TensorFlow and PyTorch. Keras was integrated into TensorFlow in 2019; Preferred Networks transitioned Chainer to PyTorch in 2019; and Microsoft stopped actively developing CNTK in 2022 to support PyTorch on Windows. -![PyTorch vs TensorFlow: Features and Functions](contents/frameworks/images/png/tensorflowpytorch.png){#tensorflow-pytorch} +![PyTorch vs. TensorFlow: Features and Functions. Source: [K&C](https://www.google.com/url?sa=i&url=https%3A%2F%2Fkruschecompany.com%2Fpytorch-vs-tensorflow%2F&psig=AOvVaw1-DSFxXYprQmYH7Z4Nk6Tk&ust=1722533288351000&source=images&cd=vfe&opi=89978449&ved=0CBEQjRxqFwoTCPDhst7m0YcDFQAAAAAdAAAAABAg)](images/png/tensorflowpytorch.png){#fig-tensorflow-pytorch} ![Popularity of ML frameworks in the United States as measured by Google web searches. Source: Google.](images/png/image6.png){#fig-ml-framework} @@ -374,9 +374,9 @@ x = torch.randn(4,784) y = torch.matmul(x, weights) + biases ``` -The above example does not have separate compile/build/run phases. Ops define and execute immediately. With dynamic graphs, the definition is intertwined with execution, providing a more intuitive, interactive workflow. However, the downside is that there is less potential for optimization since the framework only sees the graph as it is built. @static-vs-dynamic demonstrates the differences between a static and dynamic computation graph. +The above example does not have separate compile/build/run phases. Ops define and execute immediately. With dynamic graphs, the definition is intertwined with execution, providing a more intuitive, interactive workflow. However, the downside is that there is less potential for optimization since the framework only sees the graph as it is built. @fig-static-vs-dynamic demonstrates the differences between a static and dynamic computation graph. -![Comparing Static and Dynamic graphs.](contents/frameworks/images/png/staticvsdynamic.png){#static-vs-dynamic} +![Comparing static and dynamic graphs. Source: [Dev](https://www.google.com/url?sa=i&url=https%3A%2F%2Fdev-jm.tistory.com%2F4&psig=AOvVaw0r1cZbZa6iImYP-fesrN7H&ust=1722533107591000&source=images&cd=vfe&opi=89978449&ved=0CBQQjhxqFwoTCLC8nYHm0YcDFQAAAAAdAAAAABAY)](images/png/staticvsdynamic.png){#fig-static-vs-dynamic} Recently, the distinction has blurred as frameworks adopt both modes. TensorFlow 2.0 defaults to dynamic graph mode while letting users work with static graphs when needed. Dynamic declaration offers flexibility and ease of use, making frameworks more user-friendly, while static graphs provide optimization benefits at the cost of interactivity. The ideal framework balances these approaches. @tbl-exec-graph compares the pros and cons of static versus dynamic execution graphs: @@ -425,9 +425,9 @@ Data loaders also support caching and prefetching strategies to optimize data de Machine learning frameworks like TensorFlow and PyTorch provide tools to simplify and streamline the process of data augmentation, enhancing the efficiency of expanding datasets synthetically. These frameworks offer integrated functionalities to apply random transformations, such as flipping, cropping, rotating, altering color, and adding noise for images. For audio data, common augmentations involve mixing clips with background noise or modulating speed, pitch, and volume. -By integrating augmentation tools into the data pipeline, frameworks enable these transformations to be applied on the fly during each training epoch. This approach increases the variation in the training data distribution, thereby reducing overfitting and improving model generalization. @overfitting demonstrates the cases of overfitting and underfitting. The use of performant data loaders in combination with extensive augmentation capabilities allows practitioners to efficiently feed massive, varied datasets to neural networks. +By integrating augmentation tools into the data pipeline, frameworks enable these transformations to be applied on the fly during each training epoch. This approach increases the variation in the training data distribution, thereby reducing overfitting and improving model generalization. @fig-overfitting demonstrates the cases of overfitting and underfitting. The use of performant data loaders in combination with extensive augmentation capabilities allows practitioners to efficiently feed massive, varied datasets to neural networks. -![ Overfitting vs. Underfitting ](contents/frameworks/images/png/overfittingunderfitting.png){#overfitting} +![Overfitting versus underfitting. Source: [Aquarium Learning](https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.aquariumlearning.com%2Fblog-posts%2Fto-make-your-model-better-first-figure-out-whats-wrong&psig=AOvVaw3FodMJATpeLeeSsuQZBD51&ust=1722534629114000&source=images&cd=vfe&opi=89978449&ved=0CBEQjRxqGAoTCNiU49br0YcDFQAAAAAdAAAAABCoAQ)](images/png/overfittingunderfitting.png){#fig-overfitting} These hands-off data pipelines represent a significant improvement in usability and productivity. They allow developers to focus more on model architecture and less on data wrangling when training deep learning models. @@ -487,10 +487,9 @@ Visualization tools provide insight into models: * Projection - Reduce dimensionality for intuitive visualization. -* Precision-recall curves - Assess classification tradeoffs. @precision-recall shows an example of a precision-recall curve. - -![Reading a precision-recall curve.](contents/frameworks/images/png/precisionrecall.png){#precision-recall} +* Precision-recall curves - Assess classification tradeoffs. @fig-precision-recall shows an example of a precision-recall curve. +![Reading a precision-recall curve. Source: [AIM](https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.techtarget.com%2Fsearchcio%2Fdefinition%2Ftransfer-learning&psig=AOvVaw0Cbiewbu_6NsNVf314C9Q8&ust=1722534991962000&source=images&cd=vfe&opi=89978449&ved=0CBEQjRxqFwoTCPj5jITt0YcDFQAAAAAdAAAAABAE)](images/png/precisionrecall.png){#fig-precision-recall} Tools like [TensorBoard](https://www.tensorflow.org/tensorboard/scalars_and_keras) for TensorFlow and [TensorWatch](https://github.com/microsoft/tensorwatch) for PyTorch enable real-time metrics and visualization during training. @@ -550,10 +549,9 @@ These steps to remove barriers to entry continue to democratize machine learning Transfer learning is the practice of using knowledge gained from a pre-trained model to train and improve the performance of a model for a different task. For example, models such as MobileNet and ResNet are trained on the ImageNet dataset. To do so, one may freeze the pre-trained model, utilizing it as a feature extractor to train a much smaller model built on top of the feature extraction. One can also fine-tune the entire model to fit the new task. Machine learning frameworks make it easy to load pre-trained models, freeze specific layers, and train custom layers on top. They simplify this process by providing intuitive APIs and easy access to large repositories of [pre-trained models](https://keras.io/api/applications/). -Transfer learning has challenges, such as the modified model's inability to conduct its original tasks after transfer learning. Papers such as ["Learning without Forgetting"](https://browse.arxiv.org/pdf/1606.09282.pdf) by @li2017learning try to address these challenges and have been implemented in modern machine learning platforms. @transfer-learning simplifies the concept of transfer learning through an example. - -![Transfer learning.](contents/frameworks/images/png/transferlearning.png){#transfer-learning} +Transfer learning has challenges, such as the modified model's inability to conduct its original tasks after transfer learning. Papers such as ["Learning without Forgetting"](https://browse.arxiv.org/pdf/1606.09282.pdf) by @li2017learning try to address these challenges and have been implemented in modern machine learning platforms. @fig-transfer-learning simplifies the concept of transfer learning through an example. +![Transfer learning. Source: [Tech Target](https://www.google.com/url?sa=i&url=https%3A%2F%2Fanalyticsindiamag.com%2Fdevelopers-corner%2Fcomplete-guide-to-understanding-precision-and-recall-curves%2F&psig=AOvVaw3MosZItazJt2eermLTArjj&ust=1722534897757000&source=images&cd=vfe&opi=89978449&ved=0CBEQjRxqFwoTCIi389bs0YcDFQAAAAAdAAAAABAw)](images/png/transferlearning.png){#fig-transfer-learning} #### Federated Learning diff --git a/contents/hw_acceleration/hw_acceleration.qmd b/contents/hw_acceleration/hw_acceleration.qmd index 70c71ebd..d86f1c17 100644 --- a/contents/hw_acceleration/hw_acceleration.qmd +++ b/contents/hw_acceleration/hw_acceleration.qmd @@ -696,11 +696,11 @@ The [benchmarking chapter](../benchmarking/benchmarking.qmd) explores this topic Benchmarking suites such as MLPerf, Fathom, and AI Benchmark offer a set of standardized tests that can be used across different hardware platforms. These suites measure AI accelerator performance across various neural networks and machine learning tasks, from basic image classification to complex language processing. Providing a common ground for Comparison, they help ensure that performance claims are consistent and verifiable. These "tools" are applied not only to guide the development of hardware but also to ensure that the software stack leverages the full potential of the underlying architecture. -* **MLPerf:** Includes a broad set of benchmarks covering both training [@mattson2020mlperf] and inference [@reddi2020mlperf] for a range of machine learning tasks. @ml-perf showcases the uses of MLperf. +* **MLPerf:** Includes a broad set of benchmarks covering both training [@mattson2020mlperf] and inference [@reddi2020mlperf] for a range of machine learning tasks. @fig-ml-perf showcases the uses of MLperf. * **Fathom:** Focuses on core operations in deep learning models, emphasizing their execution on different architectures [@adolf2016fathom]. * **AI Benchmark:** Targets mobile and consumer devices, assessing AI performance in end-user applications [@ignatov2018ai]. -![Knowledge Distillation.](contents/optimizations/images/png/kdistillation.png){#k-dist} +![MLPerf Training v3.0 and its uses. Source: [Forbes](https://www.forbes.com/sites/stevemcdowell/2023/06/27/nvidia-h100-dominates-new-mlperf-v30-benchmark-results/)](images/png/mlperf.png){#fig-ml-perf} Benchmarks also have performance metrics that are the quantifiable measures used to evaluate the effectiveness of AI accelerators. These metrics provide a comprehensive view of an accelerator's capabilities and are used to guide the design and selection process for AI systems. Common metrics include: @@ -949,9 +949,9 @@ As a result, optical computing is still in the very early research stage despite Quantum computers leverage unique phenomena of quantum physics, like superposition and entanglement, to represent and process information in ways not possible classically. Instead of binary bits, the fundamental unit is the quantum bit or qubit. Unlike classical bits, which are limited to 0 or 1, qubits can exist simultaneously in a superposition of both states due to quantum effects. -Multiple qubits can also be entangled, leading to exponential information density but introducing probabilistic results. Superposition enables parallel computation on all possible states, while entanglement allows nonlocal correlations between qubits. @qubit simulates the structure of a qubit. +Multiple qubits can also be entangled, leading to exponential information density but introducing probabilistic results. Superposition enables parallel computation on all possible states, while entanglement allows nonlocal correlations between qubits. @fig-qubit simulates the structure of a qubit. -![Qubits, the building blocks of quantum computing.](contents/hw_acceleration/images/png/qubit.png){#qubit} +![Qubits, the building blocks of quantum computing. Source: [Microsoft](https://azure.microsoft.com/en-gb/resources/cloud-computing-dictionary/what-is-a-qubit)](images/png/qubit.png){#fig-qubit} Quantum algorithms carefully manipulate these inherently quantum mechanical effects to solve problems like optimization or search more efficiently than their classical counterparts in theory. @@ -968,9 +968,9 @@ However, quantum states are fragile and prone to errors that require error-corre While meaningful quantum advantage for ML remains far off, active research at companies like [D-Wave](https://www.dwavesys.com/company/about-d-wave/), [Rigetti](https://www.rigetti.com/), and [IonQ](https://ionq.com/) is advancing quantum computer engineering and quantum algorithms. Major technology companies like Google, [IBM](https://www.ibm.com/quantum?utm_content=SRCWW&p1=Search&p4C700050385964705&p5=e&gclid=Cj0KCQjw-pyqBhDmARIsAKd9XIPD9U1Sjez_S0z5jeDDE4nRyd6X_gtVDUKJ-HIolx2vOc599KgW8gAaAv8gEALw_wcB&gclsrc=aw.ds), and Microsoft are actively exploring quantum computing. Google recently announced a 72-qubit quantum processor called [Bristlecone](https://blog.research.google/2018/03/a-preview-of-bristlecone-googles-new.html) and plans to build a 49-qubit commercial quantum system. Microsoft also has an active research program in topological quantum computing and collaborates with quantum startup [IonQ](https://ionq.com/) -Quantum techniques may first make inroads into optimization before more generalized ML adoption. Realizing quantum ML's full potential awaits major milestones in quantum hardware development and ecosystem maturity. @q-computing illustrates a comparison between quantum computing and classical computing. +Quantum techniques may first make inroads into optimization before more generalized ML adoption. Realizing quantum ML's full potential awaits major milestones in quantum hardware development and ecosystem maturity. @fig-q-computing illustrates a comparison between quantum computing and classical computing. -![Comparing Quantum Computing with Classical Computing](contents/hw_acceleration/images/png/qcomputing.png){#q-computing} +![Comparing quantum computing with classical computing. Source: [Devopedia](​​https://devopedia.org/quantum-computing)](images/png/qcomputing.png){#fig-q-computing} ## Future Trends diff --git a/contents/introduction/introduction.qmd b/contents/introduction/introduction.qmd index e3a46912..7a621dc7 100644 --- a/contents/introduction/introduction.qmd +++ b/contents/introduction/introduction.qmd @@ -8,7 +8,7 @@ bibliography: introduction.bib ## Overview -In the early 1990s, [Mark Weiser](https://en.wikipedia.org/wiki/Mark_Weiser), a pioneering computer scientist, introduced the world to a revolutionary concept that would forever change how we interact with technology. This was succintly captured in the paper he wrote on "The Computer for the 21st Century" (@fig-ubiqutous). He envisioned a future where computing would be seamlessly integrated into our environments, becoming an invisible, integral part of daily life. This vision, which he termed "ubiquitous computing," promised a world where technology would serve us without demanding our constant attention or interaction. Fast forward to today, and we find ourselves on the cusp of realizing Weiser's vision, thanks to the advent and proliferation of machine learning systems. +In the early 1990s, [Mark Weiser](https://en.wikipedia.org/wiki/Mark_Weiser), a pioneering computer scientist, introduced the world to a revolutionary concept that would forever change how we interact with technology. He envisioned a future where computing would be seamlessly integrated into our environments, becoming an invisible, integral part of daily life. This vision, which he termed "ubiquitous computing," promised a world where technology would serve us without demanding our constant attention or interaction. Fast forward to today, and we find ourselves on the cusp of realizing Weiser's vision, thanks to the advent and proliferation of machine learning systems. ![Ubiqutous computing.](images/png/21st_computer.png){#fig-ubiqutous width=50%} @@ -24,9 +24,9 @@ However, deploying machine learning systems across the computing continuum prese Furthermore, the varying computational capabilities and energy constraints of devices at different layers of the computing continuum necessitate the development of efficient and adaptable machine learning models. Techniques such as model compression, federated learning, and transfer learning can help address these challenges, enabling the deployment of intelligence across a wide range of devices. -As we move towards the realization of Weiser's vision of ubiquitous computing, the development and deployment of machine learning systems across the entire ecosystem will be critical. By leveraging the strengths of each layer of the computing continuum, we can create an intelligent environment that seamlessly integrates with our daily lives, anticipating our needs and enhancing our experiences in ways that were once unimaginable. As we continue to push the boundaries of what's possible with distributed machine learning, we inch closer to a future where technology becomes an invisible but integral part of our world. @applications-of-ml illustrates some common applications of AI around us. +As we move towards the realization of Weiser's vision of ubiquitous computing, the development and deployment of machine learning systems across the entire ecosystem will be critical. By leveraging the strengths of each layer of the computing continuum, we can create an intelligent environment that seamlessly integrates with our daily lives, anticipating our needs and enhancing our experiences in ways that were once unimaginable. As we continue to push the boundaries of what's possible with distributed machine learning, we inch closer to a future where technology becomes an invisible but integral part of our world. @fig-applications-of-ml illustrates some common applications of AI around us. -![Common applications of Machine Learning](contents/introduction/images/cloud-edge-tiny.png){#applications-of-ml} +![Common applications of Machine Learning. Source: [EDUCBA](https://www.educba.com/applications-of-machine-learning/)](images/png/mlapplications.png){#fig-applications-of-ml} ## What's Inside the Book @@ -34,15 +34,15 @@ In this book, we will explore the technical foundations of ubiquitous machine le To improve the learning experience, we have included a variety of supplementary materials. Throughout the book, you will find slides that summarize key concepts, videos that provide in-depth explanations and demonstrations, exercises that reinforce your understanding, and labs that offer hands-on experience with the tools and techniques discussed. These additional resources are designed to cater to different learning styles and help you gain a deeper, more practical understanding of the subject matter. -We begin with the fundamentals, introducing key concepts in systems and machine learning, and providing a deep learning primer. We then guide you through the AI workflow, from data engineering to selecting the right AI frameworks. The training section covers efficient AI training techniques, model optimizations, and AI acceleration using specialized hardware. Deployment is addressed next, with chapters on benchmarking AI, distributed learning, and ML operations. Advanced topics like security, privacy, responsible AI, sustainable AI, robust AI, and generative AI are then explored in depth. The book concludes by highlighting the positive impact of AI and its potential for good. @ml-project-lifecycle outlines the lifecycle of a machine leanring project. +We begin with the fundamentals, introducing key concepts in systems and machine learning, and providing a deep learning primer. We then guide you through the AI workflow, from data engineering to selecting the right AI frameworks. The training section covers efficient AI training techniques, model optimizations, and AI acceleration using specialized hardware. Deployment is addressed next, with chapters on benchmarking AI, distributed learning, and ML operations. Advanced topics like security, privacy, responsible AI, sustainable AI, robust AI, and generative AI are then explored in depth. The book concludes by highlighting the positive impact of AI and its potential for good. @fig-ml-lifecycle outlines the lifecycle of a machine learning project. -![Machine Learning Project Life Cycle](contents/introduction/images/png/mlapplications.png){#ml-project-lifecycle} +![Machine Learning project life cycle. Source:[Medium](https://ihsanulpro.medium.com/complete-machine-learning-project-flowchart-explained-0f55e52b9381)](images/png/mlprojectlifecycle.png){#fig-ml-lifecycle} ## How to Navigate This Book To get the most out of this book, we recommend a structured learning approach that leverages the various resources provided. Each chapter includes slides, videos, exercises, and labs to cater to different learning styles and reinforce your understanding. Additionally, an AI tutor bot (SocratiQ AI) is readily available to guide you through the content and provide personalized assistance. -1. **Fundamentals (Chapters 1-3):** Start by building a strong foundation with the initial chapters, which provide an introduction to embedded AI and cover core topics like embedded systems and deep learning. +1. **Fundamentals (Chapters 1-3):** Start by building a strong foundation with the initial chapters, which provide an introduction to AI and cover core topics like AI systems and deep learning. 2. **Workflow (Chapters 4-6):** With that foundation, move on to the chapters focused on practical aspects of the AI model building process like workflows, data engineering, and frameworks. @@ -54,21 +54,21 @@ To get the most out of this book, we recommend a structured learning approach th 6. **Social Impact (Chapter 19):** Explore the positive applications and potential of AI for societal good. -7. **Conclusion (Chapter 20):** Reflect on the key takeaways and future directions in embedded AI. +7. **Conclusion (Chapter 20):** Reflect on the key takeaways and future directions in AI systems. While the book is designed for progressive learning, we encourage an interconnected learning approach that allows you to navigate chapters based on your interests and needs. Throughout the book, you'll find case studies and hands-on exercises that help you relate theory to real-world applications. We also recommend participating in forums and groups to engage in [discussions](https://github.com/harvard-edge/cs249r_book/discussions), debate concepts, and share insights with fellow learners. Regularly revisiting chapters can help reinforce your learning and offer new perspectives on the concepts covered. By adopting this structured yet flexible approach and actively engaging with the content and the community, you'll embark on a fulfilling and enriching learning experience that maximizes your understanding. ## Chapter Breakdown -Here's a closer look at what each chapter covers. We have structured the book into six main sections: Fundamentals, Workflow, Training, Deployment, Advanced Topics, and Impact. These sections closely reflect the major components of a typical machine learning pipeline, from understanding the basic concepts to deploying and maintaining AI systems in real-world applications. By organizing the content in this manner, we aim to provide a logical progression that mirrors the actual process of developing and implementing embedded AI solutions. +Here's a closer look at what each chapter covers. We have structured the book into six main sections: Fundamentals, Workflow, Training, Deployment, Advanced Topics, and Impact. These sections closely reflect the major components of a typical machine learning pipeline, from understanding the basic concepts to deploying and maintaining AI systems in real-world applications. By organizing the content in this manner, we aim to provide a logical progression that mirrors the actual process of developing and implementing AI systems. ### Fundamentals -In the Fundamentals section, we lay the groundwork for understanding embedded AI. We introduce key concepts, provide an overview of machine learning systems, and dive into the principles and algorithms of deep learning that power AI applications in embedded systems. This section equips you with the essential knowledge needed to grasp the subsequent chapters. +In the Fundamentals section, we lay the groundwork for understanding AI. This is far from being a thorough deep dive into the algorithms, but we aim to introduce key concepts, provide an overview of machine learning systems, and dive into the principles and algorithms of deep learning that power AI applications in their associated systems. This section equips you with the essential knowledge needed to grasp the subsequent chapters. -1. **[Introduction:](../introduction/introduction.qmd)** This chapter sets the stage, providing an overview of embedded AI and laying the groundwork for the chapters that follow. +1. **[Introduction:](../introduction/introduction.qmd)** This chapter sets the stage, providing an overview of AI and laying the groundwork for the chapters that follow. 2. **[ML Systems:](../ml_systems/ml_systems.qmd)** We introduce the basics of machine learning systems, the platforms where AI algorithms are widely applied. -3. **[Deep Learning Primer:](../dl_primer/dl_primer.qmd)** This chapter offers a comprehensive introduction to the algorithms and principles that underpin AI applications in embedded systems. +3. **[Deep Learning Primer:](../dl_primer/dl_primer.qmd)** This chapter offers a brief introduction to the algorithms and principles that underpin AI applications in ML systems. ### Workflow @@ -80,26 +80,26 @@ The Workflow section guides you through the practical aspects of building AI mod ### Training -In the Training section, we explore techniques for training efficient and reliable AI models. We cover strategies for achieving efficiency, model optimizations, and the role of specialized hardware in AI acceleration. This section empowers you with the knowledge to develop high-performing models that can be seamlessly integrated into embedded systems. +In the Training section, we explore techniques for training efficient and reliable AI models. We cover strategies for achieving efficiency, model optimizations, and the role of specialized hardware in AI acceleration. This section empowers you with the knowledge to develop high-performing models that can be seamlessly integrated into AI systems. 7. **[AI Training:](../training/training.qmd)** This chapter explores model training, exploring techniques for developing efficient and reliable models. 8. **[Efficient AI:](../efficient_ai/efficient_ai.qmd)** Here, we discuss strategies for achieving efficiency in AI applications, from computational resource optimization to performance enhancement. -9. **[Model Optimizations:](../optimizations/optimizations.qmd)** We explore various avenues for optimizing AI models for seamless integration into embedded systems. -10. **[AI Acceleration:](../hw_acceleration/hw_acceleration.qmd)** We discuss the role of specialized hardware in enhancing the performance of embedded AI systems. +9. **[Model Optimizations:](../optimizations/optimizations.qmd)** We explore various avenues for optimizing AI models for seamless integration into AI systems. +10. **[AI Acceleration:](../hw_acceleration/hw_acceleration.qmd)** We discuss the role of specialized hardware in enhancing the performance of AI systems. ### Deployment -The Deployment section focuses on the challenges and solutions for deploying AI models on embedded devices. We discuss benchmarking methods to evaluate AI system performance, techniques for on-device learning to improve efficiency and privacy, and the processes involved in ML operations. This section equips you with the skills to effectively deploy and maintain AI functionalities in embedded systems. +The Deployment section focuses on the challenges and solutions for deploying AI models. We discuss benchmarking methods to evaluate AI system performance, techniques for on-device learning to improve efficiency and privacy, and the processes involved in ML operations. This section equips you with the skills to effectively deploy and maintain AI functionalities in AI systems. 11. **[Benchmarking AI:](../benchmarking/benchmarking.qmd)** This chapter focuses on how to evaluate AI systems through systematic benchmarking methods. 12. **[On-Device Learning:](../ondevice_learning/ondevice_learning.qmd)** We explore techniques for localized learning, which enhances both efficiency and privacy. -13. **[ML Operations:](../ops/ops.qmd)** This chapter looks at the processes involved in the seamless integration, monitoring, and maintenance of AI functionalities in embedded systems. +13. **[ML Operations:](../ops/ops.qmd)** This chapter looks at the processes involved in the seamless integration, monitoring, and maintenance of AI functionalities. ### Advanced Topics -In the Advanced Topics section, We will study the critical issues surrounding embedded AI. We address privacy and security concerns, explore the ethical principles of responsible AI, discuss strategies for sustainable AI development, examine techniques for building robust AI models, and introduce the exciting field of generative AI. This section broadens your understanding of the complex landscape of embedded AI and prepares you to navigate its challenges. +In the Advanced Topics section, We will study the critical issues surrounding AI. We address privacy and security concerns, explore the ethical principles of responsible AI, discuss strategies for sustainable AI development, examine techniques for building robust AI models, and introduce the exciting field of generative AI. This section broadens your understanding of the complex landscape of AI and prepares you to navigate its challenges. -14. **[Security & Privacy:](../privacy_security/privacy_security.qmd)** As AI becomes more ubiquitous, this chapter addresses the crucial aspects of privacy and security in embedded AI systems. +14. **[Security & Privacy:](../privacy_security/privacy_security.qmd)** As AI becomes more ubiquitous, this chapter addresses the crucial aspects of privacy and security in AI systems. 15. **[Responsible AI:](../responsible_ai/responsible_ai.qmd)** We discuss the ethical principles guiding the responsible use of AI, focusing on fairness, accountability, and transparency. 16. **[Sustainable AI:](../sustainable_ai/sustainable_ai.qmd)** This chapter explores practices and strategies for sustainable AI, ensuring long-term viability and reduced environmental impact. 17. **[Robust AI:](../robust_ai/robust_ai.qmd)** We discuss techniques for developing reliable and robust AI models that can perform consistently across various conditions. @@ -107,15 +107,15 @@ In the Advanced Topics section, We will study the critical issues surrounding em ### Social Impact -The Impact section highlights the transformative potential of embedded AI in various domains. We showcase real-world applications of TinyML in healthcare, agriculture, conservation, and other areas where AI is making a positive difference. This section inspires you to leverage the power of embedded AI for societal good and to contribute to the development of impactful solutions. +The Impact section highlights the transformative potential of AI in various domains. We showcase real-world applications of TinyML in healthcare, agriculture, conservation, and other areas where AI is making a positive difference. This section inspires you to leverage the power of AI for societal good and to contribute to the development of impactful solutions. 19. **[AI for Good:](../ai_for_good/ai_for_good.qmd)** We highlight positive applications of TinyML in areas like healthcare, agriculture, and conservation. ### Closing -In the Closing section, we reflect on the key learnings from the book and look ahead to the future of embedded AI. We synthesize the concepts covered, discuss emerging trends, and provide guidance on continuing your learning journey in this rapidly evolving field. This section leaves you with a comprehensive understanding of embedded AI and the excitement to apply your knowledge in innovative ways. +In the Closing section, we reflect on the key learnings from the book and look ahead to the future of AI. We synthesize the concepts covered, discuss emerging trends, and provide guidance on continuing your learning journey in this rapidly evolving field. This section leaves you with a comprehensive understanding of AI and the excitement to apply your knowledge in innovative ways. -20. **[Conclusion:](../conclusion/conclusion.qmd)** The book concludes with a reflection on the key learnings and future directions in the field of embedded AI. +20. **[Conclusion:](../conclusion/conclusion.qmd)** The book concludes with a reflection on the key learnings and future directions in the field of AI. ## Contribute Back diff --git a/contents/ml_systems/ml_systems.qmd b/contents/ml_systems/ml_systems.qmd index 1fd02d03..d677bc43 100644 --- a/contents/ml_systems/ml_systems.qmd +++ b/contents/ml_systems/ml_systems.qmd @@ -96,7 +96,7 @@ Cloud ML fosters a collaborative environment that enables teams to work together **Cost-Effectiveness** -Adopting Cloud ML can be a cost-effective solution for organizations, especially compared to building and maintaining an on-premises machine learning infrastructure. Cloud service providers offer flexible pricing models, such as pay-as-you-go or subscription-based plans, allowing organizations to pay only for the resources they consume. This eliminates the need for upfront capital investments in hardware and infrastructure, reducing the overall cost of implementing machine learning projects. Additionally, the scalability of Cloud ML ensures that organizations can optimize their resource usage and avoid overprovisioning, further enhancing cost-efficiency. +Adopting Cloud ML can be a cost-effective solution for organizations, especially compared to building and maintaining an on-premises machine learning infrastructure. Cloud service providers offer flexible pricing models, such as pay-as-you-go or subscription-based plans, allowing organizations to pay only for the resources they consume. This eliminates the need for upfront capital investments in hardware and infrastructure, reducing the overall cost of implementing machine learning projects. Additionally, the scalability of Cloud ML ensures that organizations can optimize their resource usage and avoid over provisioning, further enhancing cost-efficiency. The benefits of Cloud ML, including its immense computational power, dynamic scalability, access to advanced tools and algorithms, collaborative environment, and cost-effectiveness, make it a compelling choice for organizations looking to harness the potential of machine learning. By leveraging the capabilities of the cloud, organizations can accelerate their machine learning initiatives, drive innovation, and gain a competitive edge in today's data-driven landscape. @@ -148,9 +148,9 @@ Cloud ML is deeply integrated into our online experiences, shaping the way we in **Security and Anomaly Detection** -Cloud ML plays a role in bolstering user security by powering anomaly detection systems. These systems continuously monitor user activities and system logs to identify unusual patterns or suspicious behavior. By analyzing vast amounts of data in real-time, Cloud ML algorithms can detect potential cyber threats, such as unauthorized access attempts, malware infections, or data breaches. The cloud's scalability and processing power enable these systems to handle the increasing complexity and volume of security data, providing a proactive approach to protecting users and systems from potential threats. @cloud-ml provides an overview of this section. +Cloud ML plays a role in bolstering user security by powering anomaly detection systems. These systems continuously monitor user activities and system logs to identify unusual patterns or suspicious behavior. By analyzing vast amounts of data in real-time, Cloud ML algorithms can detect potential cyber threats, such as unauthorized access attempts, malware infections, or data breaches. The cloud's scalability and processing power enable these systems to handle the increasing complexity and volume of security data, providing a proactive approach to protecting users and systems from potential threats. @fig-cloud-ml provides an overview of this section. -![Section summary - Cloud ML](contents/ml_systems/images/png/cloudml.png){#cloud-ml} +![Section summary for Cloud ML.](images/png/cloudml.png){#fig-cloud-ml} ## Edge ML @@ -215,9 +215,9 @@ Edge ML plays a crucial role in efficiently managing various systems in smart ho The Industrial IoT leverages Edge ML to monitor and control complex industrial processes. Here, machine learning models can analyze data from numerous sensors in real-time, enabling predictive maintenance, optimizing operations, and enhancing safety measures. This revolution in industrial automation and efficiency is transforming manufacturing and production across various sectors. -The applicability of Edge ML is vast and not limited to these examples. Various other sectors, including healthcare, agriculture, and urban planning, are exploring and integrating Edge ML to develop innovative solutions responsive to real-world needs and challenges, heralding a new era of smart, interconnected systems. @edge-ml provides an overview of this section. +The applicability of Edge ML is vast and not limited to these examples. Various other sectors, including healthcare, agriculture, and urban planning, are exploring and integrating Edge ML to develop innovative solutions responsive to real-world needs and challenges, heralding a new era of smart, interconnected systems. @fig-edge-ml provides an overview of this section. -![Section Summary - Edge ML](contents/ml_systems/images/png/edgeml.png){#edge-ml} +![Section summary for Edge ML.](images/png/edgeml.png){#fig-edge-ml} ## Tiny ML @@ -293,15 +293,15 @@ TinyML can be employed to create anomaly detection models that identify unusual In environmental monitoring, TinyML enables real-time data analysis from various field-deployed sensors. These could range from city air quality monitoring to wildlife tracking in protected areas. Through TinyML, data can be processed locally, allowing for quick responses to changing conditions and providing a nuanced understanding of environmental patterns, crucial for informed decision-making. -In summary, TinyML serves as a trailblazer in the evolution of machine learning, fostering innovation across various fields by bringing intelligence directly to the edge. Its potential to transform our interaction with technology and the world is immense, promising a future where devices are connected, intelligent, and capable of making real-time decisions and responses. @tiny-ml provides an overview of this section. +In summary, TinyML serves as a trailblazer in the evolution of machine learning, fostering innovation across various fields by bringing intelligence directly to the edge. Its potential to transform our interaction with technology and the world is immense, promising a future where devices are connected, intelligent, and capable of making real-time decisions and responses. @fig-tiny-ml provides an overview of this section. -![Section Summary - Tiny ML](contents/ml_systems/images/png/tinyml.png){#tiny-ml} +![Section summary for Tiny ML.](images/png/tinyml.png){#fig-tiny-ml} ## Comparison -Up to this point, we've explored each of the different ML variants individually. Now, let's bring them all together for a comprehensive view. @tbl-big_vs_tiny offers a comparative analysis of Cloud ML, Edge ML, and TinyML based on various features and aspects. Additionally, @venn-diagram draws a contrast using a venn diagram. This comparison provides a clear perspective on the unique advantages and distinguishing factors, aiding in making informed decisions based on the specific needs and constraints of a given application or project. +Up to this point, we've explored each of the different ML variants individually. Now, let's bring them all together for a comprehensive view. @tbl-big_vs_tiny offers a comparative analysis of Cloud ML, Edge ML, and TinyML based on various features and aspects. Additionally, @fig-venn-diagram draws a contrast using a venn diagram. This comparison provides a clear perspective on the unique advantages and distinguishing factors, aiding in making informed decisions based on the specific needs and constraints of a given application or project. -![ML Venn diagram](contents/ml_systems/images/png/venndiagram.png){#venn-diagram} +![ML Venn diagram. Source: [arXiv](https://arxiv.org/html/2403.19076v1)](images/png/venndiagram.png){#fig-venn-diagram} +--------------------------+---------------------------------------------------------+---------------------------------------------------------+----------------------------------------------------------+ | Aspect | Cloud ML | Edge ML | TinyML | diff --git a/contents/ondevice_learning/ondevice_learning.qmd b/contents/ondevice_learning/ondevice_learning.qmd index 077a5a0e..71ebd194 100644 --- a/contents/ondevice_learning/ondevice_learning.qmd +++ b/contents/ondevice_learning/ondevice_learning.qmd @@ -239,9 +239,9 @@ Transfer learning is an ML technique in which a model developed for a particular Let's take the example of a smart sensor application that uses on-device AI to recognize objects in images captured by the device. Traditionally, this would require sending the image data to a server, where a large neural network model processes the data and sends back the results. With on-device AI, the model is stored and runs directly on-device, eliminating the need to send data to a server. -If we want to customize the model for the on-device characteristics, training a neural network model from scratch on the device would be impractical due to the limited computational resources and battery life. This is where transfer learning comes in. Instead of training a model from scratch, we can take a pre-trained model, such as a convolutional neural network (CNN) or a transformer network trained on a large dataset of images, and finetune it for our specific object recognition task. This finetuning can be done directly on the device using a smaller dataset of images relevant to the task. By leveraging the pre-trained model, we can reduce the computational resources and time required for training while still achieving high accuracy for the object recognition task. @t-learning further illustrates the benefits of transfer learning over training from scratch. +If we want to customize the model for the on-device characteristics, training a neural network model from scratch on the device would be impractical due to the limited computational resources and battery life. This is where transfer learning comes in. Instead of training a model from scratch, we can take a pre-trained model, such as a convolutional neural network (CNN) or a transformer network trained on a large dataset of images, and finetune it for our specific object recognition task. This finetuning can be done directly on the device using a smaller dataset of images relevant to the task. By leveraging the pre-trained model, we can reduce the computational resources and time required for training while still achieving high accuracy for the object recognition task. @fig-t-learning further illustrates the benefits of transfer learning over training from scratch. -![Training from scratch vs. transfer learning.](contents/ondevice_learning/images/png/transfer_learning.png){#t-learning} +![Training from scratch vs. transfer learning.](images/png/transfer_learning.png){#fig-t-learning} Transfer learning is important in making on-device AI practical by allowing us to leverage pre-trained models and finetune them for specific tasks, thereby reducing the computational resources and time required for training. The combination of on-device AI and transfer learning opens up new possibilities for AI applications that are more privacy-conscious and responsive to user needs. @@ -249,9 +249,6 @@ Transfer learning has revolutionized the way models are developed and deployed, Implementation in production scenarios can be broadly categorized into two stages: pre-deployment and post-deployment. -![Training from scratch vs. transfer learning.](contents/ondevice_learning/images/png/transfer_learning.jpeg){#transfer} - - ### Pre-Deployment Specialization In the pre-deployment stage, transfer learning acts as a catalyst to expedite the development process. Here's how it typically works: Imagine we are creating a system to recognize different breeds of dogs. Rather than starting from scratch, we can use a pre-trained model that has already mastered the broader task of recognizing animals in images. @@ -443,9 +440,9 @@ With this proposed structure, there are a few key vectors for further optimizing ::: -@federated-learning outlines the transformative impact of federated learning on on-device learning. +@fig-federated-learning outlines the transformative impact of federated learning on on-device learning. -![Federated learning is revolutionizing on-device learning.](contents/ondevice_learning/images/png/federatedvsoil.png){#federated-learning} +![Federated learning is revolutionizing on-device learning.](images/png/federatedvsoil.png){#fig-federated-learning} ### Communication Efficiency diff --git a/contents/ops/ops.qmd b/contents/ops/ops.qmd index 4f478bc2..46da88e6 100644 --- a/contents/ops/ops.qmd +++ b/contents/ops/ops.qmd @@ -762,10 +762,9 @@ Despite the proliferation of new MLOps tools in response to the increase in dema * Seamless deployment onto edge devices through compilation, SDKs, and benchmarks * Collaboration features for teams and integration with other platforms -With Edge Impulse, developers with limited data science expertise can develop specialized ML models that run efficiently within small computing environments. It provides a comprehensive solution for creating embedded intelligence and advancing machine learning. @edge-impulse further illustrates this concept. - -![The inner workings of edge impulse.](contents/ops/images/png/impulse.png){#edge-impulse} +With Edge Impulse, developers with limited data science expertise can develop specialized ML models that run efficiently within small computing environments. It provides a comprehensive solution for creating embedded intelligence and advancing machine learning. @fig-edge-impulse further illustrates this concept. +![The inner workings of edge impulse. Source: [Edge Impulse](https://www.edgeimpulse.com/blog/getting-started-with-edge-impulse/)](images/png/impulse.png){#fig-edge-impulse} ##### User Interface diff --git a/contents/optimizations/images/png/kdistillation.png b/contents/optimizations/images/png/kdistillation.png deleted file mode 100644 index 44106663..00000000 Binary files a/contents/optimizations/images/png/kdistillation.png and /dev/null differ diff --git a/contents/privacy_security/privacy_security.qmd b/contents/privacy_security/privacy_security.qmd index 5a140da2..134f4073 100644 --- a/contents/privacy_security/privacy_security.qmd +++ b/contents/privacy_security/privacy_security.qmd @@ -78,7 +78,9 @@ But Stuxnet wasn't designed to steal information or spy on people. Its goal was Stuxnet was incredibly advanced malware built by national governments to cross from the digital realm into real-world infrastructure. It specifically targeted important industrial machines, where embedded machine learning is highly applicable in a way never done before. The virus provided a wake-up call about how sophisticated cyberattacks could now physically destroy equipment and facilities. -This breach was significant due to its sophistication; Stuxnet specifically targeted programmable logic controllers (PLCs) used to automate electromechanical processes such as the speed of centrifuges for uranium enrichment. The worm exploited vulnerabilities in the Windows operating system to gain access to the Siemens Step7 software controlling the PLCs. Despite not being a direct attack on ML systems, Stuxnet is relevant for all embedded systems as it showcases the potential for state-level actors to design attacks that bridge the cyber and physical worlds with devastating effects. +This breach was significant due to its sophistication; Stuxnet specifically targeted programmable logic controllers (PLCs) used to automate electromechanical processes such as the speed of centrifuges for uranium enrichment. The worm exploited vulnerabilities in the Windows operating system to gain access to the Siemens Step7 software controlling the PLCs. Despite not being a direct attack on ML systems, Stuxnet is relevant for all embedded systems as it showcases the potential for state-level actors to design attacks that bridge the cyber and physical worlds with devastating effects. @fig-stuxnet explains Stuxnet in greater detail. + +![Stuxnet explained. Source: [IEEE Spectrum](https://spectrum.ieee.org/the-real-story-of-stuxnet)](images/png/stuxnet.png){#fig-stuxnet} ### Jeep Cherokee Hack @@ -918,7 +920,9 @@ These examples demonstrate how federated learning provides tangible privacy bene #### Core Idea -Machine unlearning is a fairly new process that describes how the influence of a subset of training data can be removed from the model. Several methods have been used to perform machine unlearning and remove the influence of a subset of training data from the final model. A baseline approach might consist of simply fine-tuning the model for more epochs on just the data that should be remembered to decrease the influence of the data "forgotten" by the model. Since this approach doesn't explicitly remove the influence of data that should be erased, membership inference attacks are still possible, so researchers have adopted other approaches to unlearn data from a model explicitly. One type of approach that researchers have adopted includes adjusting the model loss function to treat the losses of the "forget set explicitly" (data to be unlearned) and the "retain set" (remaining data that should still be remembered) differently [@tarun2023deep; @khan2021knowledgeadaptation]. +Machine unlearning is a fairly new process that describes how the influence of a subset of training data can be removed from the model. Several methods have been used to perform machine unlearning and remove the influence of a subset of training data from the final model. A baseline approach might consist of simply fine-tuning the model for more epochs on just the data that should be remembered to decrease the influence of the data "forgotten" by the model. Since this approach doesn't explicitly remove the influence of data that should be erased, membership inference attacks are still possible, so researchers have adopted other approaches to unlearn data from a model explicitly. One type of approach that researchers have adopted includes adjusting the model loss function to treat the losses of the "forget set explicitly" (data to be unlearned) and the "retain set" (remaining data that should still be remembered) differently [@tarun2023deep; @khan2021knowledgeadaptation]. @fig-machine-unlearning illustrates some of the applications of Machine-unlearning. + +![Applications of Machine Unlearning. Source: [BBVA OpenMind](https://www.bbvaopenmind.com/en/technology/artificial-intelligence/ai-and-machine-unlearning-forgotten-path/)](images/png/machineunlearning.png){#fig-machine-unlearning} #### Case Study @@ -940,7 +944,7 @@ Homomorphic encryption is a form of encryption that allows computations to be ca Homomorphic encryption enables outsourced computation on encrypted data without exposing the data itself to the external party performing the operations. However, only certain computations like addition and multiplication are supported in partially homomorphic schemes. Fully homomorphic encryption (FHE) that can handle any computation is even more complex. The number of possible operations is limited before noise accumulation corrupts the ciphertext. -To use homomorphic encryption across different entities, carefully generated public keys must be exchanged for operations across separately encrypted data. This advanced encryption technique enables previously impossible secure computation paradigms but requires expertise to implement correctly for real-world systems. +To use homomorphic encryption across different entities, carefully generated public keys must be exchanged for operations across separately encrypted data. This advanced encryption technique enables previously impossible secure computation paradigms but requires expertise to implement correctly for real-world systems. #### Benefits diff --git a/contents/sustainable_ai/sustainable_ai.qmd b/contents/sustainable_ai/sustainable_ai.qmd index 4df21708..5a721446 100644 --- a/contents/sustainable_ai/sustainable_ai.qmd +++ b/contents/sustainable_ai/sustainable_ai.qmd @@ -49,9 +49,9 @@ Additionally, there is an increasing need for AI companies to scrutinize their c Furthermore, more than voluntary self-regulation may be needed---governments may need to introduce new regulations aimed at sustainable AI standards and practices if we hope to curb the projected energy explosion of ever-larger models. Reported metrics like computing usage, carbon footprint, and efficiency benchmarks could hold organizations accountable. -Through ethical principles, company policies, and public rules, AI technologists and corporations have a profound duty to our planet to ensure the responsible and sustainable advancement of technology positioned to transform modern society radically. We owe it to future generations to get this right. @ethical-ai outlines some ethical concerns and challenges facing AI. +Through ethical principles, company policies, and public rules, AI technologists and corporations have a profound duty to our planet to ensure the responsible and sustainable advancement of technology positioned to transform modern society radically. We owe it to future generations to get this right. @fig-ethical-ai outlines some ethical concerns and challenges facing AI. -![Ethical challenges in AI.](contents/sustainable_ai/images/png/ethicalai.png){#ethical-ai} +![Ethical challenges in AI development. Source: [COE](https://www.coe.int/en/web/bioethics/common-ethical-challenges-in-ai)](images/png/ethicalai.png){#fig-ethical-ai} ### Long-term Sustainability {#long-term-sustainability} diff --git a/contents/workflow/workflow.html b/contents/workflow/workflow.html new file mode 100644 index 00000000..38415726 --- /dev/null +++ b/contents/workflow/workflow.html @@ -0,0 +1,1515 @@ + + + + + + + + + +4  AI Workflow – Machine Learning Systems + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + +
+ +
+ + +
+ + + +
+ +
+
+

4  AI Workflow

+
+ + + +
+ + + + +
+ + + +
+ + +

Resources: Slides, Videos, Exercises, Labs

+
+
+

+
DALL·E 3 Prompt: Create a rectangular illustration of a stylized flowchart representing the AI workflow/pipeline. From left to right, depict the stages as follows: ‘Data Collection’ with a database icon, ‘Data Preprocessing’ with a filter icon, ‘Model Design’ with a brain icon, ‘Training’ with a weight icon, ‘Evaluation’ with a checkmark, and ‘Deployment’ with a rocket. Connect each stage with arrows to guide the viewer horizontally through the AI processes, emphasizing these steps’ sequential and interconnected nature.
+
+
+

In this chapter, we’ll explore the machine learning (ML) workflow, setting the stage for subsequent chapters that go deeper into the specifics. To ensure we see the bigger picture, this chapter offers a high-level overview of the steps involved in the ML workflow.

+

The ML workflow is a structured approach that guides professionals and researchers through developing, deploying, and maintaining ML models. This workflow is generally divided into several crucial stages, each contributing to the effective development of intelligent systems.

+
+
+
+ +
+
+Learning Objectives +
+
+
+
    +

  • Understand the ML workflow and gain insights into the structured approach and stages of developing, deploying, and maintaining machine learning models.

    • +

  • Learn about the unique challenges and distinctions between workflows for Traditional machine learning and embedded AI.

    • +

  • Appreciate the roles in ML projects and understand their responsibilities and significance.

    • +

  • Understanding the importance, applications, and considerations for implementing ML models in resource-constrained environments.

    • +

  • Gain awareness about the ethical and legal aspects that must be considered and adhered to in ML and embedded AI projects.

    • +

  • Establish a basic understanding of ML workflows and roles to be well-prepared for deeper exploration in the following chapters.

    • +
+
+
+
+

4.1 Overview

+
+
+
+ +
+
+Figure 4.1: Multi-step design methodology for the development of a machine learning model. Commonly referred to as the machine learning lifecycle +
+
+
+

Developing a successful machine learning model requires a systematic workflow. This end-to-end process enables you to build, deploy, and maintain models effectively. As shown in Figure fig-ml-life-cycle, It typically involves the following key steps:

+
    +
  1. Problem Definition - Start by clearly articulating the specific problem you want to solve. This focuses on your efforts during data collection and model building.
    2. +
  2. Data Collection and Preparation: Gather relevant, high-quality training data that captures all aspects of the problem. Clean and preprocess the data to prepare it for modeling.
    3. +
  3. Model Selection and Training: Choose a machine learning algorithm suited to your problem type and data. Consider the pros and cons of different approaches. Feed the prepared data into the model to train it. Training time varies based on data size and model complexity.
    4. +
  4. Model Evaluation: Test the trained model on new unseen data to measure its predictive accuracy. Identify any limitations.
    5. +
  5. Model Deployment: Integrate the validated model into applications or systems to start operationalization.
    6. +
  6. Monitor and Maintain: Track model performance in production. Retrain periodically on new data to keep it current.
    7. +
+

Following this structured ML workflow helps guide you through the key phases of development. It ensures you build effective and robust models ready for real-world deployment, resulting in higher-quality models that solve your business needs.

+

The ML workflow is iterative, requiring ongoing monitoring and potential adjustments. Additional considerations include:

+
    +
  • Version Control: Track code and data changes to reproduce results and revert to earlier versions if needed.
    • +
  • Documentation: Maintain detailed documentation for workflow understanding and reproduction.
    • +
  • Testing: Rigorously test the workflow to ensure its functionality.
    • +
  • Security: Safeguard your workflow and data when deploying models in production settings.
    • +
+
+
+

4.2 Traditional vs. Embedded AI

+

The ML workflow is a universal guide applicable across various platforms, including cloud-based solutions, edge computing, and TinyML. However, the workflow for Embedded AI introduces unique complexities and challenges, making it a captivating domain and paving the way for remarkable innovations. Figure fig-dl-and-ml illustrates the differences between Machine Learning and Deep Learning.

+
+
+
+ +
+
+Figure 4.2: Comparing traditional Machine Learning and Deep Learning. Source: BBN Times +
+
+
+

Figure fig-embedded-ai showcases the uses of embedded ai in various industries.

+
+
+
+ +
+
+Figure 4.3: Embedded AI applications. Source: Rinf.tech +
+
+
+
+

4.2.1 Resource Optimization

+
    +
  • Traditional ML Workflow: This workflow prioritizes model accuracy and performance, often leveraging abundant computational resources in cloud or data center environments.
    • +
  • Embedded AI Workflow: Given embedded systems’ resource constraints, this workflow requires careful planning to optimize model size and computational demands. Techniques like model quantization and pruning are crucial.
    • +
+
+
+

4.2.2 Real-time Processing

+
    +
  • Traditional ML Workflow: Less emphasis on real-time processing, often relying on batch data processing.
    • +
  • Embedded AI Workflow: Prioritizes real-time data processing, making low latency and quick execution essential, especially in applications like autonomous vehicles and industrial automation.
    • +
+
+
+

4.2.3 Data Management and Privacy

+
    +
  • Traditional ML Workflow: Processes data in centralized locations, often necessitating extensive data transfer and focusing on data security during transit and storage.
    • +
  • Embedded AI Workflow: This workflow leverages edge computing to process data closer to its source, reducing data transmission and enhancing privacy through data localization.
    • +
+
+
+

4.2.4 Hardware-Software Integration

+
    +
  • Traditional ML Workflow: Typically operates on general-purpose hardware, with software development occurring independently.
    • +
  • Embedded AI Workflow: This workflow involves a more integrated approach to hardware and software development, often incorporating custom chips or hardware accelerators to achieve optimal performance.
    • +
+
+
+
+

4.3 Roles & Responsibilities

+

Creating an ML solution, especially for embedded AI, is a multidisciplinary effort involving various specialists. Unlike traditional software development, building an ML solution demands a multidisciplinary approach due to the experimental nature of model development and the resource-intensive requirements of training and deploying these models.

+

There is a pronounced need for roles focusing on data for the success of machine learning pipelines. Data scientists and data engineers handle data collection, build data pipelines, and ensure data quality. Since the nature of machine learning models depends on the data they consume, the models are unique and vary with different applications, necessitating extensive experimentation. Machine learning researchers and engineers drive this experimental phase through continuous testing, validation, and iteration to achieve optimal performance.

+

The deployment phase often requires specialized hardware and infrastructure, as machine learning models can be resource-intensive, demanding high computational power and efficient resource management. This necessitates collaboration with hardware engineers to ensure that the infrastructure can support the computational demands of model training and inference.

+

As models make decisions that can impact individuals and society, ethical and legal aspects of machine learning are becoming increasingly important. Ethicists and legal advisors are needed to ensure compliance with ethical standards and legal regulations.

+

Table tbl-mlops_roles shows a rundown of the typical roles involved. While the lines between these roles can sometimes blur, the table below provides a general overview.

+
+
+
+Table 4.1: Roles and responsibilities of people involved in MLOps. +
+
+ ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RoleResponsibilities
Project ManagerOversees the project, ensuring timelines and milestones are met.
Domain ExpertsOffer domain-specific insights to define project requirements.
Data ScientistsSpecialize in data analysis and model development.
Machine Learning EngineersFocus on model development and deployment.
Data EngineersManage data pipelines.
Embedded Systems EngineersIntegrate ML models into embedded systems.
Software DevelopersDevelop software components for AI system integration.
Hardware EngineersDesign and optimize hardware for the embedded AI system.
UI/UX DesignersFocus on user-centric design.
QA EngineersEnsure the system meets quality standards.
Ethicists and Legal AdvisorsConsult on ethical and legal compliance.
Operations and Maintenance PersonnelMonitor and maintain the deployed system.
Security SpecialistsEnsure system security.
+
+
+
+

Understanding these roles is crucial for completing an ML project. As we proceed through the upcoming chapters, we’ll explore each role’s essence and expertise, fostering a comprehensive understanding of the complexities involved in embedded AI projects. This holistic view facilitates seamless collaboration and nurtures an environment ripe for innovation and breakthroughs.

+
+
+

4.4 Conclusion

+

This chapter has laid the foundation for understanding the machine learning workflow, a structured approach crucial for the development, deployment, and maintenance of ML models. By exploring the distinct stages of the ML lifecycle, we have gained insights into the unique challenges faced by traditional ML and embedded AI workflows, particularly in terms of resource optimization, real-time processing, data management, and hardware-software integration. These distinctions underscore the importance of tailoring workflows to meet the specific demands of the application environment.

+

The chapter emphasized the significance of multidisciplinary collaboration in ML projects. Understanding the diverse roles provides a comprehensive view of the teamwork necessary to navigate the experimental and resource-intensive nature of ML development. As we move forward to more detailed discussions in the subsequent chapters, this high-level overview equips us with a holistic perspective on the ML workflow and the various roles involved.

+
+
+

4.5 Resources

+

Here is a curated list of resources to support students and instructors in their learning and teaching journeys. We are continuously working on expanding this collection and will add new exercises soon.

+
+
+
+ +
+
+Slides +
+
+
+
+
+

These slides are a valuable tool for instructors to deliver lectures and for students to review the material at their own pace. We encourage students and instructors to leverage these slides to improve their understanding and facilitate effective knowledge transfer.

+ +
+
+
+
+
+
+ +
+
+Videos +
+
+
+
+
+
    +
  • Coming soon.
    • +
+
+
+
+
+
+
+ +
+
+Exercises +
+
+
+
+
+

To reinforce the concepts covered in this chapter, we have curated a set of exercises that challenge students to apply their knowledge and deepen their understanding.

+
    +
  • Coming soon.
    • +
+
+
+
+
+
+
+ +
+
+Labs +
+
+
+
+
+

In addition to exercises, we offer a series of hands-on labs allowing students to gain practical experience with embedded AI technologies. These labs provide step-by-step guidance, enabling students to develop their skills in a structured and supportive environment. We are excited to announce that new labs will be available soon, further enriching the learning experience.

+
    +
  • Coming soon.
    • +
+
+
+
+ + +
+ +
+ + + + + +
+ + + + + + \ No newline at end of file diff --git a/contents/workflow/workflow.qmd b/contents/workflow/workflow.qmd index 5580526e..fab73039 100644 --- a/contents/workflow/workflow.qmd +++ b/contents/workflow/workflow.qmd @@ -56,13 +56,13 @@ The ML workflow is iterative, requiring ongoing monitoring and potential adjustm ## Traditional vs. Embedded AI -The ML workflow is a universal guide applicable across various platforms, including cloud-based solutions, edge computing, and TinyML. However, the workflow for Embedded AI introduces unique complexities and challenges, making it a captivating domain and paving the way for remarkable innovations. @dl-and-ml illustrates the differences between Machine Learning and Deep Learning. +The ML workflow is a universal guide applicable across various platforms, including cloud-based solutions, edge computing, and TinyML. However, the workflow for Embedded AI introduces unique complexities and challenges, making it a captivating domain and paving the way for remarkable innovations. @fig-dl-and-ml illustrates the differences between Machine Learning and Deep Learning. -![Comparing Traditional Machine Learning and Deep Learning.](contents/workflow/images/png/comparingdlandml.png){#dl-and-ml} +![Comparing traditional Machine Learning and Deep Learning. Source: [BBN Times](https://www.bbntimes.com/technology/to-leverage-deep-learning-you-must-know-this-first)](images/png/comparingdlandml.png){#fig-dl-and-ml} -@embedded-ai showcases the uses of embedded ai in various industries. +@fig-embedded-ai showcases the uses of embedded ai in various industries. -![Embedded AI applications](contents/workflow/images/png/embeddedai.png){#embedded-ai} +![Embedded AI applications. Source: [Rinf.tech](https://www.rinf.tech/what-is-embedded-intelligence-and-how-can-tech-leaders-embrace-it/)](images/png/embeddedai.png){#fig-embedded-ai} ### Resource Optimization