Skip to content

Commit

Permalink
Merge pull request #103 from gtbook/frank_jan19
Browse files Browse the repository at this point in the history
Edit sections 2.1 -> 3.3
  • Loading branch information
dellaert authored Jan 21, 2025
2 parents 9610b08 + a4cfaad commit 2b6e5a3
Show file tree
Hide file tree
Showing 49 changed files with 1,413 additions and 1,085 deletions.
85 changes: 71 additions & 14 deletions S10_introduction.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -31,27 +31,48 @@
"Moreover, society now finds itself at the dawn of a new, golden age of robotics,\n",
"as robots are increasingly moving out of highly regulated factory environments into\n",
"commercial settings, hospitals, our homes, our streets, and even the skies.\n",
"How do these systems work? What sensors do they have and how do they perceive our world through these senses? What are the mechanisms by which they can move and act in their environment, and how do they choose their actions at any given moment? \n",
"How do these systems work? What sensors do they have and how do they perceive our world through these senses?\n",
"What are the mechanisms by which they can move and act in their environment, and how do they choose their actions at any given moment?\n",
"How can they learn to adapt their behavior over time to improve their capabilities and enhance interactions with humans they encounter?\n",
"These questions frame an incredibly broad and rich subject, and given the recent advances in artificial intelligence the field of robotics is bound to see many breakthroughs over the next few years.\n",
"These questions frame an incredibly broad and rich subject, and given the recent advances in artificial intelligence,\n",
"the field of robotics is bound to see many breakthroughs over the next few years.\n",
"\n",
"This introductory text is designed to guide the reader through the *foundations* of modern robotics, while at the same time showing how practical systems can be implemented. The theoretical concepts in each chapter are explained informally without sacrificing rigor, balanced with a unique hands-on approach. Theory is immediately put into practice, as every section in the book is interspersed with example code and can be executed as a python notebook, allowing the reader to experiment with implementations that are representative of many robotics systems. No experience is needed beyond a good basis in math, especially linear algebra and notions of statistics, and some knowledge of the python programming language. The code itself is based on popular open source libraries, most notably GTSAM and PyTorch, which provide a good starting point for those who want to explore robotics as a potential career choice."
"This introductory text is designed to guide the reader through the *foundations* of modern robotics,\n",
"while at the same time showing how practical systems can be implemented.\n",
"The theoretical concepts in each chapter are explained informally without sacrificing rigor,\n",
"balanced with a unique hands-on approach. Theory is immediately put into practice,\n",
"as every section in the book is interspersed with example code and can be executed as a python notebook,\n",
"allowing the reader to experiment with implementations that are representative of many robotics systems.\n",
"No experience is needed beyond a good basis in math, especially linear algebra and notions of statistics,\n",
"and some knowledge of the python programming language.\n",
"The code itself is based on popular open source libraries, most notably GTSAM and PyTorch,\n",
"which provide a good starting point for those who want to explore robotics as a potential career choice."
]
},
{
"cell_type": "markdown",
"id": "Ow_ar2AAFHi0",
"metadata": {},
"source": [
"```{index} sense-think-act\n",
"```\n",
"\n",
"The sense-think-act cycle is a recurring theme in this book.\n",
"While the range of applications for modern robotic systems is quite diverse, at a high level, most all systems take the same approach.\n",
"They use sensors and perception algorithms to assess their environment; they reason about their goals and how to achieve them; they perform actions to change the state of the world.\n",
"They use sensors and perception algorithms to assess their environment; they reason about their goals and how to achieve them; they perform actions to change the state of the world.\n",
"This is known as the **sense-think-act** cycle.\n",
"The robot senses its world, thinks about what to do, then acts.\n",
"This idea is general enough to be applied at many levels of abstraction and time scales.\n",
"We can imagine a warehouse robot tasked with retrieving an item from inventory. After using its sensors to determine its current location (sense),\n",
"the robot constructs a path (think), then traverses the path (act). The time scale here might be on the order of seconds to minutes to complete one iteration of the sense-think-act loop, and the level of abstraction is very high, ignoring details like controlling motors.\n",
"On the other hand, we could consider a surgical robot tasked with precisely inserting a needle into a tumor. In this case, ultrasonic sensors could be used to localize the needle relative to the tumor (sense), small corrections to needle placement would be computed (think), and a low-level motion command would be sent to the device, possibly a voltage or current, to effect the motion (act). Here, the time scale is on the order of milliseconds, and the level of abstraction could be quite detailed, including tissue dynamics and flexibility of the needle.\n",
"The sense-think-act cycle is a recurring theme in this book."
"the robot constructs a path (think), then traverses the path (act).\n",
"The time scale here might be on the order of seconds to minutes to complete one iteration of the sense-think-act cycle,\n",
"and the level of abstraction is very high, ignoring details like controlling motors.\n",
"On the other hand, we could consider a surgical robot tasked with precisely inserting a needle into a tumor.\n",
"In this case, ultrasonic sensors could be used to localize the needle relative to the tumor (sense),\n",
"small corrections to needle placement would be computed (think),\n",
"and a low-level motion command would be sent to the device, possibly a voltage or current, to effect the motion (act).\n",
"Here, the time scale is on the order of milliseconds, and the level of abstraction could be quite detailed,\n",
"including tissue dynamics and flexibility of the needle."
]
},
{
Expand All @@ -62,8 +83,16 @@
"```{index} differential drive robot, DDR\n",
"```\n",
"\n",
" The chapters of the book proceed through a sequence of increasingly complex robotic systems.\n",
" We begin, in Chapter 2, by considering a simple trash sorting robot. Pieces of trash arrive on a conveyor, and the robot’s task is to place these in an appropriate bin. For this first robotic system, we abstract away most all details that would confront a real trash sorting robot, focusing our attention on fundamental concepts from probability theory, both to solve a simple perception problem (categorizing pieces of trash) and to solve a planning problem (which bin to select). From there, in each chapter we incrementally add complexity to the system models and the mathematical methods for manipulating them, developing increasingly sophisticated algorithms to implement these methods. We consider a simple vacuum cleaning robot in Chapter 3, a mobile robot that navigates in a warehouse in Chapter 4, a two-wheeled, differential drive robot (DDR) in Chapter 5, self-driving cars in Chapter 6, and quadrotor drones in Chapter 7."
"The chapters of the book proceed through a sequence of increasingly complex robotic systems.\n",
"We begin, in Chapter 2, by considering a simple trash-sorting robot. Pieces of trash arrive on a conveyor,\n",
"and the robot’s task is to place these in an appropriate bin.\n",
"For this first robotic system, we abstract away most details that would confront a real trash-sorting robot,\n",
"focusing our attention on fundamental concepts from probability theory,\n",
"both to solve a simple perception problem (categorizing pieces of trash) and to solve a planning problem (which bin to select).\n",
"From there, in each chapter we incrementally add complexity to the system models and the mathematical methods for manipulating them,\n",
"developing increasingly sophisticated algorithms to implement these methods.\n",
"We consider a simple vacuum cleaning robot in Chapter 3, a mobile robot that navigates in a warehouse in Chapter 4,\n",
"a two-wheeled, differential-drive robot (DDR) in Chapter 5, self-driving cars in Chapter 6, and quadrotor drones in Chapter 7."
]
},
{
Expand All @@ -74,8 +103,18 @@
"```{index} graphical models\n",
"```\n",
"\n",
"The individual chapters share a common structure, delving into how to represent the robot's state, how to describe its actuators and sensors, the corresponding perception and planning methods and algorithms, and ending with a section on learning from data. The expressiveness of the state space, which is tied to how much we do or do not abstract away from reality, determines to a large degree the complexity of the corresponding algorithms. By gradually building up this richness, we ensure that each chapter builds upon the previous one, guiding the reader along a series of achievable steps. Key to our approach is the use of **graphical models** to help explain how state, actions, and sensing share many commonalities across a diverse set of autonomous robots.\n",
"This organization also captures the essense of the sense-think-act cycle. Each chapter contains sections on sensors and perception (sense), planning and learning (think), and state and action models (act)."
"The individual chapters share a common structure, examining how to represent the robot's state,\n",
"how to describe its actuators and sensors, the associated perception and planning methods and algorithms,\n",
"and concluding with a section on learning from data.\n",
"The expressiveness of the state space, which depends on how much we abstract away from reality,\n",
"largely determines the complexity of the corresponding algorithms.\n",
"By gradually increasing this complexity, we ensure that each chapter builds on the previous one,\n",
"guiding the reader through a series of manageable steps.\n",
"A key element of our approach is the use of **graphical models** to illustrate how state, actions,\n",
"and sensing share common patterns across a wide range of autonomous robots.\n",
"This structure also reflects the essence of the sense-think-act cycle.\n",
"Each chapter includes sections on sensors and perception (sense), planning and learning (think),\n",
"and state and action models (act)."
]
},
{
Expand All @@ -86,7 +125,17 @@
"```{index} state, sensing, action, SE(2), special Euclidean group, SE(3)\n",
"```\n",
"\n",
"In the state, action, and sensing sections of each chapter we focus on the mathematical *models*. To model robots, we move from discrete state spaces and fairly abstract action and sensing, to continuous state spaces and real-world sensing in later sections. We introduce the special Euclidean group of order two, $SE(2)$ for differential drive robots, and extend it to $SE(3)$ for the case of drones. We develop the differential kinematics for wheeled mobile robots, both for omnidirectional robots (our logistics robot), and for robots with nonholonomic constraints. In the case of drones, we derive a model for the dynamics that includes propellor thrust and aerodynamic properties such as drag. We take a similar approach with sensors. In Chapter 2, we introduce simple sensors that measure physical properties of a piece of trash. Then, in subsequent chapters, we consider an increasing array of sensors, including proximity sensors, RFID beacons, GPS, cameras, LIDAR, and Inertial Measurement Units (IMUs)."
"In the state, action, and sensing sections of each chapter, we focus on the mathematical *models*.\n",
"To model robots, we progress from discrete state spaces with abstract action and sensing\n",
"to continuous state spaces with real-world sensing in later sections.\n",
"We introduce the special Euclidean group of order two, $SE(2)$, for differential-drive robots\n",
"and extend it to $SE(3)$ for drones.\n",
"We develop the differential kinematics for wheeled mobile robots, both for omnidirectional robots (our logistics robot)\n",
"and for robots with nonholonomic constraints.\n",
"For drones, we derive a dynamics model that includes propeller thrust and aerodynamic effects such as drag.\n",
"We take a similar approach with sensors.\n",
"In Chapter 2, we introduce simple sensors that measure physical properties of a piece of trash.\n",
"In later chapters, we expand to a variety of sensors, including proximity sensors, RFID beacons, GPS, cameras, LIDAR, and Inertial Measurement Units (IMUs)."
]
},
{
Expand All @@ -97,9 +146,17 @@
"```{index} maximum likelihood, maximum a posteriori\n",
"```\n",
"\n",
"In other places, we focus on probabilistic methods for representing and reasoning about uncertainty. We introduce discrete probability distributions for simple classification problems, and develop maximum-likelihood and maximum-a-posteriori estimation methods for these using our trash-sorting robot. We introduce continuous distributions, and specifically the uniform and Gaussian distributions, to model the motion of mobile robots and the effects of sensor noise. To reason about the effects of uncertainty, we introduce sampling-based methods, such as particle filtering and Markov localization, Kalman filters, Bayes nets, and Markov decision processes. In each case, we begin by formulating the appropriate mathematical models, and then proceed to describe algorithmic solutions (often illustrated with code).\n",
"In other places, we focus on probabilistic methods for representing and reasoning about uncertainty.\n",
"We introduce discrete probability distributions for simple classification problems\n",
"and develop maximum-likelihood and maximum-a-posteriori estimation methods using our trash-sorting robot.\n",
"We then move to continuous distributions, specifically the uniform and Gaussian distributions,\n",
"to model the motion of mobile robots and the effects of sensor noise.\n",
"To reason about uncertainty, we introduce sampling-based methods such as particle filtering and Markov localization,\n",
"along with Kalman filters, Bayes nets, and Markov decision processes.\n",
"In each case, we start by formulating the appropriate mathematical models,\n",
"then proceed to describe algorithmic solutions, often illustrated with code.\n",
"\n",
"In the remainder of this chapter, we give a slightly more detailed overview of the topics covered in this book."
"In the remainder of this chapter, we provide a slightly more detailed overview of the topics covered in this book."
]
}
],
Expand Down
6 changes: 5 additions & 1 deletion S11_models.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,11 @@
"attachments": {},
"cell_type": "markdown",
"id": "ba24d9f2",
"metadata": {},
"metadata": {
"tags": [
"no-pdf"
]
},
"source": [
"<img src=\"Figures1/S11-Robot_menagerie-09.jpg\" alt=\"Splash image with robot pondering state\" width=\"40%\" align=center style=\"vertical-align:middle;margin:10px 0px\">"
]
Expand Down
14 changes: 12 additions & 2 deletions S12_reasoning.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,18 @@
"source": [
"# Reasoning\n",
"\n",
"> Reasoning provides the *intelligence* of so-called intelligent robotic systems.\n",
"\n",
"> Reasoning provides the *intelligence* of so-called intelligent robotic systems."
]
},
{
"cell_type": "markdown",
"id": "693d5cd9",
"metadata": {
"tags": [
"no-pdf"
]
},
"source": [
"<img src=\"Figures1/S12-Robot_menagerie-03.jpg\" alt=\"Splash image with thinking robot\" width=\"40%\" align=center style=\"vertical-align:middle;margin:10px 0px\">"
]
},
Expand Down
14 changes: 12 additions & 2 deletions S13_math.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,18 @@
"source": [
"# The Mathematics of Robotics\n",
"\n",
"> Probability theory and statistics, linear algebra, and optimization are key mathematical tools for modern robotics.\n",
"\n",
"> Probability theory and statistics, linear algebra, and optimization are key mathematical tools for modern robotics."
]
},
{
"cell_type": "markdown",
"id": "c12dd9b0",
"metadata": {
"tags": [
"no-pdf"
]
},
"source": [
"<img src=\"Figures1/S15-Robot_menagerie-02.jpg\" alt=\"Splash image with robot thinking deeply\" width=\"40%\" align=center style=\"vertical-align:middle;margin:10px 0px\">"
]
},
Expand Down
22 changes: 4 additions & 18 deletions S20_sorter_intro.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"\n",
"In this chapter we introduce the main concepts that will be covered in this book using\n",
"a simple trash sorting robot as an example.\n",
"Imagine, if you will, a trash sorting facility that has installed a robot at one of its conveyer belts to help out with the sorting of trash, in order to increase the proportion of trash that can be sustainably recycled. Imagine also that you are tasked with creating the software that will govern the robot's actions. How would you go about this? Even this simple robot will be sufficiently rich to tease out the main sense-think-act cycle that govern most if not all robot systems.\n",
"Imagine, if you will, a trash sorting facility that has installed a robot at one of its conveyor belts to help out with the sorting of trash, in order to increase the proportion of trash that can be sustainably recycled. Imagine also that you are tasked with creating the software that will govern the robot's actions. How would you go about this? Even this simple robot will be sufficiently rich to tease out the main sense-think-act cycle that governs most, if not all, robot systems.\n",
"\n",
"Let us think about the robot's workings and role in a bit more detail. The robot is presented with items of trash, one at a time, on a conveyor belt.\n",
"Depending on the *category of the item* (state), e.g., paper, metal, etc.,\n",
Expand All @@ -35,29 +35,15 @@
"The robot can *refine its model of the world based on data* acquired during operation (learning),\n",
"and this new model can be used to improve operation over time.\n",
"\n",
"For now, we will abstract away many details of real-world robotics problems. \n",
"For now, we will abstract away many details of real-world robotics problems.\n",
"We will not consider\n",
"the geometric or physical aspects of robot motion, and we will assume that the robot is\n",
"able to move items from the conveyor belt to the commanded bin without error.\n",
"On the other hand, we will explicitly consider uncertainty associated to the problem\n",
"On the other hand, we will explicitly consider uncertainty associated with the problem\n",
"of classifying items of trash into their appropriate categories, and how this uncertainty\n",
"affects decision making when choosing actions (i.e., destination bins for items of trash).\n",
"This will allow us to introduce some basic concepts from probability and decision theory\n",
"that will be used throughout the book.\n",
"\n",
"<!-- \n",
"\n",
"- State:\t\tplastic|paper|cardboard|metal|glass|unknown\n",
"- Actuators: \tgripper (4 actions or no-op)\n",
"- Sensors: Attributes -> conductivity (2), computer vision (3), weight (continuous)\n",
"- Perception: \tEstimation probabilities of plastic|paper|cardboard|metal|unknown\n",
"- Planning: \tDecision Theory -> sort into three bins (plastics|mixed paper|metals|glass) or do nothing\n",
"\n",
"\n",
"\n",
"Perception is different from sensing: it turns prior knowledge and sensed information into a probabilistic knowledge about the current state of the world/robot.\n",
"\n",
"Using probabilistic knowledge, we can try to act optimally, and hence we will define notions of optimality, and develop some decision theory. -->"
"that will be used throughout the book."
]
},
{
Expand Down
Loading

0 comments on commit 2b6e5a3

Please sign in to comment.