Skip to content

Commit 24dcca1

Browse files
authored
[docs] Add detector modes section to guide (#331)
Add a section to our docs for Count- and Multi-class- detectors.
1 parent cb3fd95 commit 24dcca1

10 files changed

+254
-42
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Binary Classification Detectors
2+
3+
Binary classification detectors are used to answer yes/no questions about images. Most of Groundlight's documentation examples are for binary classification detectors, as they are the simplest type of detector.
4+
5+
In order to create a binary classification detector, you need to provide a query that asks a yes/no question. For example, "Is there an eagle visible?" or "Is the door fully closed?".
6+
7+
```python notest
8+
from groundlight import Groundlight
9+
gl = Groundlight()
10+
11+
# highlight-start
12+
detector = gl.create_detector(
13+
name="eagle-detector",
14+
query="Is there an eagle visible?",
15+
confidence_threshold=0.9,
16+
)
17+
# highlight-end
18+
```
19+
20+
## Submit an Image Query to a Binary Classification Detector
21+
22+
Now that you have created a binary classification detector, you can submit an image query to it.
23+
24+
```python notest
25+
from groundlight import Groundlight
26+
gl = Groundlight()
27+
28+
detector = gl.get_detector_by_name("eagle-detector")
29+
30+
# highlight-start
31+
# Check if an eagle is visible in an image
32+
image_query = gl.submit_image_query(detector, "path/to/image.jpg")
33+
# highlight-end
34+
35+
print(f"Result: {image_query.result.label}")
36+
print(f"Confidence: {image_query.result.confidence}")
37+
```
38+
39+
Binary classification detectors return a `label` attribute in the result object, which will be either `"YES"` or `"NO"`. If a query is ambiguous, it is also possible for the detector to return an `"UNCLEAR"` label.
40+
41+
The `confidence` attribute represents the confidence level in the predicted label, which (for a binary classification detector) is a value between 0.5 and 1. A higher confidence score indicates that the model is more certain about its prediction.
42+
43+
## Add a label to a Binary Classification Detector
44+
45+
To provide ground truth labels for binary classification detectors, you can specify the label as either `"YES"`, `"NO"`, or `"UNCLEAR"`. This helps improve the accuracy of your detector over time.
46+
47+
```python notest
48+
from groundlight import Groundlight
49+
gl = Groundlight()
50+
51+
# highlight-start
52+
# Add a binary label to the image query from the previous example
53+
gl.add_label(image_query, label="YES")
54+
# highlight-end
55+
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Multiple Choice (Choose One) Detectors
2+
3+
If you want to classify images into multiple categories, you can create a multi-class detector.
4+
5+
```python notest
6+
from groundlight import ExperimentalApi
7+
gl_exp = ExperimentalApi()
8+
9+
# highlight-start
10+
class_names = ["Golden Retriever", "Labrador Retriever", "German Shepherd", "Other"]
11+
detector = gl_exp.create_multiclass_detector(
12+
name="dog-breed-detector",
13+
query="What kind of dog is this?",
14+
class_names=class_names,
15+
)
16+
# highlight-end
17+
```
18+
19+
:::tip
20+
We recommend adding an "Other" class to your multi-class detector to handle cases where the image does not belong to any of the pre-defined classes.
21+
:::
22+
23+
:::note
24+
Multi-Class Detectors are available on [Business and Enterprise plans](https://www.groundlight.ai/pricing).
25+
:::
26+
27+
## Submit an Image Query to a Multi-Class Detector
28+
29+
Now that you have created a multi-class detector, you can submit an image query to it.
30+
31+
```python notest
32+
from groundlight import ExperimentalApi
33+
gl_exp = ExperimentalApi()
34+
35+
detector = gl_exp.get_detector_by_name("dog-breed-detector")
36+
37+
# highlight-start
38+
# Classify the breed of a dog in an image
39+
image_query = gl_exp.submit_image_query(detector, "path/to/image.jpg")
40+
# highlight-end
41+
42+
print(f"Result: {image_query.result.label}")
43+
print(f"Confidence: {image_query.result.confidence}")
44+
```
45+
46+
Multi-class detectors return a `label` attribute in the result object, which contains the predicted class label. The `label` attribute will be one of the class names provided when creating the detector. The `confidence` attribute represents the confidence level in the predicted class, which is a value between `1/len(class_names)` and 1.
47+
48+
## Add a label to a Multi-Class Detector
49+
50+
To provide ground truth labels for multi-class detectors, you can specify the label of the correct class.
51+
52+
```python notest
53+
from groundlight import ExperimentalApi
54+
gl_exp = ExperimentalApi()
55+
56+
# highlight-start
57+
# Add a multi-class label to the image query from the previous example
58+
gl_exp.add_label(image_query, label="German Shepherd")
59+
# highlight-end
60+
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# Count Detectors
2+
3+
Counting detectors are used to count the number of objects in an image. Groundlight's counting detectors also return bounding boxes around the objects they count.
4+
5+
```python notest
6+
from groundlight import ExperimentalApi
7+
gl_exp = ExperimentalApi()
8+
9+
# highlight-start
10+
detector = gl_exp.create_counting_detector(
11+
name="car-counter",
12+
query="How many cars are in the parking lot?",
13+
class_name="car",
14+
max_count=20,
15+
confidence_threshold=0.2,
16+
)
17+
# highlight-end
18+
```
19+
20+
Counting detectors should be provided with a query that asks "how many" objects are in the image.
21+
22+
A maximum count (of 25 or fewer) must be specified when creating a counting detector. This is the maximum number of objects that the detector will count in an image. Groundlight's ML models are optimized for counting up to 20 objects, but you can increase the maximum count to 25 if needed. If you have an application that requires counting more than 25 objects, please [contact us](mailto:[email protected]).
23+
24+
The `confidence_threshold` parameter sets the minimum confidence level required for the ML model's predictions. If the model's confidence falls below this threshold, the query will be sent for human review. Count detectors can have a `confidence_threshold` set to any value between `1/(max_count + 2)` and 1.
25+
26+
:::note
27+
Counting Detectors are available on [Business and Enterprise plans](https://www.groundlight.ai/pricing).
28+
:::
29+
30+
## Submit an Image Query to a Counting Detector
31+
32+
Now that you have created a counting detector, you can submit an image query to it.
33+
34+
```python notest
35+
from groundlight import ExperimentalApi
36+
gl_exp = ExperimentalApi()
37+
38+
detector = gl_exp.get_detector_by_name("car-counter")
39+
40+
# highlight-start
41+
# Count the number of cars in an image
42+
image_query = gl_exp.submit_image_query(detector, "path/to/image.jpg")
43+
# highlight-end
44+
45+
print(f"Counted {image_query.result.count} cars")
46+
print(f"Confidence: {image_query.result.confidence}")
47+
print(f"Bounding Boxes: {image_query.rois}")
48+
```
49+
50+
In the case of counting detectors, the `count` attribute of the result object will contain the number of objects counted in the image. The `confidence` attribute represents the confidence level in the specific count. Note that this implies that confidences may be lower (on average) for counting detectors with a higher maximum count.
51+
52+
<!-- TODO: display an example image with bounding boxes -->
53+
54+
:::tip Drawing Bounding Boxes
55+
You can visualize the bounding boxes returned by counting detectors using a library like OpenCV. Here's an example of how to draw bounding boxes on an image:
56+
57+
```python notest
58+
import cv2
59+
import numpy as np
60+
61+
def draw_bounding_boxes(image_path, rois):
62+
"""
63+
Draw bounding boxes on an image based on ROIs returned from a counting detector.
64+
65+
Args:
66+
image_path: Path to the image file
67+
rois: List of ROI objects returned from image_query.rois
68+
"""
69+
image = cv2.imread(image_path)
70+
if image is None:
71+
raise ValueError(f"Could not read image from {image_path}")
72+
height, width = image.shape[:2]
73+
74+
# Draw bounding boxes
75+
for roi in rois:
76+
x1 = int(roi.geometry.left * width)
77+
y1 = int(roi.geometry.top * height)
78+
x2 = int(roi.geometry.right * width)
79+
y2 = int(roi.geometry.bottom * height)
80+
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
81+
label_text = f"{roi.label}: {roi.score:.2f}"
82+
cv2.putText(image, label_text, (x1, y1-10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
83+
84+
# Display the image
85+
cv2.imshow("Image with Bounding Boxes", image)
86+
cv2.waitKey(0)
87+
cv2.destroyAllWindows()
88+
89+
# Example usage:
90+
# image_query = gl.submit_image_query(detector, "path/to/image.jpg")
91+
# draw_bounding_boxes("path/to/image.jpg", image_query.rois)
92+
```
93+
:::
94+
95+
## Add a label to a Counting Detector
96+
97+
The Groundlight API allows you to add labels to image queries, including Region of Interest (ROI) data.
98+
When adding a label to a counting detector, if you include ROIs, the number of ROIs should match
99+
the count you are labeling.
100+
101+
```python notest
102+
from groundlight import ExperimentalApi
103+
gl_exp = ExperimentalApi()
104+
105+
# highlight-start
106+
# Add a count label with corresponding ROIs to the image query from the previous example.
107+
# ROIs are specified as (left, top) and (right, bottom) coordinates, with values
108+
# between 0 and 1 representing the percentage of the image width and height.
109+
roi1 = gl_exp.create_roi("car", (0.1, 0.2), (0.2, 0.3))
110+
roi2 = gl_exp.create_roi("car", (0.4, 0.4), (0.5, 0.6))
111+
roi3 = gl_exp.create_roi("car", (0.6, 0.5), (0.8, 0.9))
112+
rois = [roi1, roi2, roi3]
113+
gl_exp.add_label(image_query, label=len(rois), rois=rois)
114+
# highlight-end
115+
```
+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"label": "Answer Modes",
3+
"position": 3
4+
}
+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Detector Answer Modes
2+
3+
Groundlight offers several detector modalities to suit different computer vision tasks. While previous examples have focused on binary classification, this guide will walk you through using counting and multi-class detectors. Let's explore how these different modes can be used via the Groundlight SDK.
4+
5+
- **[Binary Detectors](1-binary-detectors.md)**: Learn how to create detectors that answer yes/no questions about images.
6+
- **[Multiple Choice (Choose One) Detectors](2-multi-choice-detectors.md)**: Create detectors that select one answer from a predefined list of options.
7+
- **[Count Detectors](3-counting-detectors.md)**: Use detectors to count the number of objects present in an image - and return bounding boxes around the counted objects.
8+
<!-- 4. [Text Recognition Detectors](4-text-recognition-detectors.md) -->
9+
10+
<!-- TODO: object detection modes -->
+6-38
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
# Working with Detectors
22

3-
### Explicitly create a new detector
3+
This guide will walk you through creating, retrieving, and managing detectors in Groundlight. Groundlight supports several detector modalities to suit different computer vision tasks - for more information on these modes, see the [Detector Answer Modes](../answer-modes/answer-modes.md) guide.
4+
5+
6+
## Explicitly create a new detector
47

58
Typically you'll use the `get_or_create_detector(name: str, query: str)` method to find an existing detector you've already created with the same name, or create a new one if it doesn't exists. But if you'd like to force creating a new detector you can also use the `create_detector(name: str, query: str)` method
69

@@ -16,7 +19,7 @@ detector = gl.create_detector(name="your_detector_name", query="is there a hummi
1619
# highlight-end
1720
```
1821

19-
### Retrieve an existing detector
22+
## Retrieve an existing detector
2023
To work with a detector that you've previously created, you need to retrieve it using its unique identifier. This is typical in Groundlight applications where you want to continue to use a detector you've already created.
2124

2225
<!-- Don't test because the ID can't be faked -->
@@ -43,7 +46,7 @@ detector = gl.get_detector_by_name(name="your_detector_name")
4346
# highlight-end
4447
```
4548

46-
### List your detectors
49+
## List your detectors
4750
To manage and interact with your detectors, you might need to list them. Groundlight provides a straightforward way to retrieve a list of detectors you've created. By default, the list is paginated to show 10 results per page, but you can customize this to suit your needs.
4851

4952
```python
@@ -59,38 +62,3 @@ detectors = gl.list_detectors()
5962
detectors = gl.list_detectors(page=1, page_size=5)
6063
# highlight-end
6164
```
62-
63-
### [BETA] Create a Counting Detector
64-
So far, all of the detectors we've created have been binary classification detectors. But what if you want to count the number of objects in an image? You can create a counting detector to do just that. Counting detectors also return bounding boxes around the objects they count.
65-
66-
:::note
67-
68-
Counting Detectors are available on [Pro, Business, and Enterprise plans](https://www.groundlight.ai/pricing).
69-
70-
:::
71-
72-
```python notest
73-
from groundlight import ExperimentalApi
74-
75-
gl_experimental = ExperimentalApi()
76-
77-
# highlight-start
78-
detector = gl_experimental.create_counting_detector(name="your_detector_name", query="How many cars are in the parking lot?", max_count=20)
79-
# highlight-end
80-
```
81-
82-
### [BETA] Create a Multi-Class Detector
83-
If you want to classify images into multiple categories, you can create a multi-class detector.
84-
85-
```python notest
86-
from groundlight import ExperimentalApi
87-
88-
gl_experimental = ExperimentalApi()
89-
90-
# highlight-start
91-
class_names = ["Golden Retriever", "Labrador Retriever", "German Shepherd"]
92-
detector = gl_experimental.create_multiclass_detector(
93-
name, query="What kind of dog is this?", class_names=class_names
94-
)
95-
# highlight-end
96-
```

docs/docs/guide/_category_.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
{
22
"label": "Guide",
3-
"position": 3
3+
"position": 4
44
}

docs/docs/guide/guide.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,6 @@ On the following pages, we'll guide you through the process of building applicat
1111
- **[Asynchronous queries](7-async-queries.md)**: Groundlight makes it easy to submit asynchronous queries. Learn how to submit queries asynchronously and retrieve the results later.
1212
- **[Using Groundlight on the edge](8-edge.md)**: Discover how to deploy Groundlight in edge computing environments for improved performance and reduced latency.
1313
- **[Alerts](9-alerts.md)**: Learn how to set up alerts to notify you via text (SMS) or email when specific conditions are met in your visual applications.
14-
- **[Industrial applications](../sample-applications/4-industrial.md)**: Learn how to apply modern natural-language-based computer vision to your industrial and manufacturing applications.
14+
1515

1616
By exploring these resources and sample applications, you'll be well on your way to building powerful visual applications using Groundlight's computer vision and natural language capabilities.
+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
22
"label": "Alternative Deployment Options",
3-
"position": 5,
3+
"position": 6,
44
"collapsed": false
55
}
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
{
22
"label": "Sample Applications",
3-
"position": 4
3+
"position": 5
44
}

0 commit comments

Comments
 (0)