Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Add detector modes section to guide #331

Merged
merged 11 commits into from
Mar 19, 2025
Merged
File renamed without changes.
35 changes: 0 additions & 35 deletions docs/docs/guide/3-working-with-detectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,38 +59,3 @@ detectors = gl.list_detectors()
detectors = gl.list_detectors(page=1, page_size=5)
# highlight-end
```

### [BETA] Create a Counting Detector
So far, all of the detectors we've created have been binary classification detectors. But what if you want to count the number of objects in an image? You can create a counting detector to do just that. Counting detectors also return bounding boxes around the objects they count.

:::note

Counting Detectors are available on [Pro, Business, and Enterprise plans](https://www.groundlight.ai/pricing).

:::

```python notest
from groundlight import ExperimentalApi

gl_experimental = ExperimentalApi()

# highlight-start
detector = gl_experimental.create_counting_detector(name="your_detector_name", query="How many cars are in the parking lot?", max_count=20)
# highlight-end
```

### [BETA] Create a Multi-Class Detector
If you want to classify images into multiple categories, you can create a multi-class detector.

```python notest
from groundlight import ExperimentalApi

gl_experimental = ExperimentalApi()

# highlight-start
class_names = ["Golden Retriever", "Labrador Retriever", "German Shepherd"]
detector = gl_experimental.create_multiclass_detector(
name, query="What kind of dog is this?", class_names=class_names
)
# highlight-end
```
2 changes: 1 addition & 1 deletion docs/docs/guide/4-submitting-image-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ image_query = gl.get_image_query(id=image_query.id)
# highlight-end
```

See this [guide](./7-async-queries.md) for more information on ImageQueries submitted asynchronously.
See this [guide](./8-async-queries.md) for more information on ImageQueries submitted asynchronously.

### (Advanced) Get the first available answer, regardless of confidence
`ask_ml` evaluates an image with Groundlight and returns the first answer Groundlight can provide, agnostic of confidence. There is no wait period when using this method. It is called `ask_ml` because our machine learning models are earliest on our escalation ladder and thus always the fastest to respond.
Expand Down
172 changes: 172 additions & 0 deletions docs/docs/guide/5-detector-modes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Detector Answer Modes
Groundlight offers several detector modalities to suit different computer vision tasks. While previous examples have focused on binary classification, this guide will walk you through using counting and multi-class detectors. Let's explore how these different modes can be used via the Groundlight SDK.

## Counting Detectors

Counting detectors are used to count the number of objects in an image. Groundlight's counting detectors also return bounding boxes around the objects they count.

```python
from groundlight import ExperimentalApi
gl_exp = ExperimentalApi()

# highlight-start
detector = gl_exp.create_counting_detector(
name="car-counter",
query="How many cars are in the parking lot?",
max_count=20,
confidence_threshold=0.2,
)
# highlight-end
```

Counting detectors should be provided with a query that asks "how many" objects are in the image.

A maximum count (of 50 or fewer) must be specified when creating a counting detector. This is the maximum number of objects that the detector will count in an image. Groundlight's ML models are optimized for counting up to 20 objects, but you can increase the maximum count to 50 if needed. If you have an application that requires counting more than 50 objects, please [contact us](mailto:[email protected]).

:::note
Counting Detectors are available on [Business and Enterprise plans](https://www.groundlight.ai/pricing).
:::

### Submit an Image Query to a Counting Detector

Now that you have created a counting detector, you can submit an image query to it.

```python notest
from groundlight import ExperimentalApi
gl_exp = ExperimentalApi()

detector = gl_exp.get_detector_by_name("car-counter")

# highlight-start
# Count the number of cars in an image
image_query = gl_exp.submit_image_query(detector, "path/to/image.jpg")
# highlight-end

print(f"Counted {image_query.result.count} cars")
print(f"Confidence: {image_query.result.confidence}")
print(f"Bounding Boxes: {image_query.rois}")
```

In the case of counting detectors, the `count` attribute of the result object will contain the number of objects counted in the image. The `confidence` attribute represents the confidence level in the specific count. Note that this implies that confidences may be lower (on average) for counting detectors with a higher maximum count.

<!-- TODO: display an example image with bounding boxes -->

:::tip Drawing Bounding Boxes
You can visualize the bounding boxes returned by counting detectors using a library like OpenCV. Here's an example of how to draw bounding boxes on an image:

```python notest
import cv2
import numpy as np

def draw_bounding_boxes(image_path, rois):
"""
Draw bounding boxes on an image based on ROIs returned from a counting detector.

Args:
image_path: Path to the image file
rois: List of ROI objects returned from image_query.rois
"""
image = cv2.imread(image_path)
if image is None:
raise ValueError(f"Could not read image from {image_path}")
height, width = image.shape[:2]

# Draw bounding boxes
for roi in rois:
x1 = int(roi.geometry.left * width)
y1 = int(roi.geometry.top * height)
x2 = int(roi.geometry.right * width)
y2 = int(roi.geometry.bottom * height)
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
label_text = f"{roi.label}: {roi.score:.2f}"
cv2.putText(image, label_text, (x1, y1-10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

# Display the image
cv2.imshow("Image with Bounding Boxes", image)
cv2.waitKey(0)
cv2.destroyAllWindows()

# Example usage:
# image_query = gl.submit_image_query(detector, "path/to/image.jpg")
# draw_bounding_boxes("path/to/image.jpg", image_query.rois)
```
:::

### Add a label to a Counting Detector

The Groundlight API allows you to add labels to image queries, including Region of Interest (ROI) data.
When adding a label to a counting detector, if you include ROIs, the number of ROIs should match
the count you are labeling.

```python notest
from groundlight import ExperimentalApi
gl_exp = ExperimentalApi()

# highlight-start
# Add a count label with corresponding ROIs to the image query from the previous example
roi1 = gl_exp.create_roi("car", (0.1, 0.2), (0.2, 0.3))
roi2 = gl_exp.create_roi("car", (0.4, 0.4), (0.5, 0.6))
roi3 = gl_exp.create_roi("car", (0.6, 0.5), (0.8, 0.9))
rois = [roi1, roi2, roi3]
gl_exp.add_label(image_query, label=len(rois), rois=rois)
# highlight-end
```

## [BETA] Multi-Class Detectors

If you want to classify images into multiple categories, you can create a multi-class detector.

```python
from groundlight import ExperimentalApi
gl_exp = ExperimentalApi()

# highlight-start
class_names = ["Golden Retriever", "Labrador Retriever", "German Shepherd", "Other"]
detector = gl_exp.create_multiclass_detector(
name="dog-breed-detector",
query="What kind of dog is this?",
class_names=class_names,
)
# highlight-end
```

:::tip
We recommend adding an "Other" class to your multi-class detector to handle cases where the image does not belong to any of the pre-defined classes.
:::

### Submit an Image Query to a Multi-Class Detector

Now that you have created a multi-class detector, you can submit an image query to it.

```python notest
from groundlight import ExperimentalApi
gl_exp = ExperimentalApi()

detector = gl_exp.get_detector_by_name("dog-breed-detector")

# highlight-start
# Classify the breed of a dog in an image
image_query = gl_exp.submit_image_query(detector, "path/to/image.jpg")
# highlight-end

print(f"Result: {image_query.result.label}")
print(f"Confidence: {image_query.result.confidence}")
```

Multi-class detectors return a `label` attribute in the result object, which contains the predicted class label. The `label` attribute will be one of the class names provided when creating the detector. The `confidence` attribute represents the confidence level in the predicted class, which is a value between `1/len(class_names)` and 1.

### Add a label to a Multi-Class Detector

To provide ground truth labels for multi-class detectors, you can specify the label of the correct class.

```python notest
from groundlight import ExperimentalApi
gl_exp = ExperimentalApi()

# highlight-start
# Add a multi-class label to the image query from the previous example
gl_exp.add_label(image_query, label="German Shepherd")
# highlight-end
```

<!-- TODO: text, object detection modes -->
File renamed without changes.
File renamed without changes.
13 changes: 7 additions & 6 deletions docs/docs/guide/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,12 @@ On the following pages, we'll guide you through the process of building applicat
- **[Grabbing images](2-grabbing-images.md)**: Understand the intricacies of how to submit images from various input sources to Groundlight.
- **[Working with detectors](3-working-with-detectors.md)**: Learn how to create, configure, and use detectors in your Groundlight-powered applications.
- **[Submitting image queries](4-submitting-image-queries.md)**: Submit images to Groundlight for analysis and retrieve the results.
- **[Confidence levels](5-managing-confidence.md)**: Master how to control the trade-off of latency against accuracy by configuring the desired confidence level for your detectors.
- **[Handling errors](6-handling-errors.md)**: Understand how to handle and troubleshoot HTTP errors (`ApiException`) that may occur while using Groundlight.
- **[Asynchronous queries](7-async-queries.md)**: Groundlight makes it easy to submit asynchronous queries. Learn how to submit queries asynchronously and retrieve the results later.
- **[Using Groundlight on the edge](8-edge.md)**: Discover how to deploy Groundlight in edge computing environments for improved performance and reduced latency.
- **[Alerts](9-alerts.md)**: Learn how to set up alerts to notify you via text (SMS) or email when specific conditions are met in your visual applications.
- **[Industrial applications](../sample-applications/4-industrial.md)**: Learn how to apply modern natural-language-based computer vision to your industrial and manufacturing applications.
- **[Detector answer modes](5-detector-modes.md)**: Answer counting and multi-classification queries with Groundlight.
- **[Confidence levels](6-managing-confidence.md)**: Master how to control the trade-off of latency against accuracy by configuring the desired confidence level for your detectors.
- **[Handling errors](7-handling-errors.md)**: Understand how to handle and troubleshoot HTTP errors (`ApiException`) that may occur while using Groundlight.
- **[Asynchronous queries](8-async-queries.md)**: Groundlight makes it easy to submit asynchronous queries. Learn how to submit queries asynchronously and retrieve the results later.
- **[Using Groundlight on the edge](9-edge.md)**: Discover how to deploy Groundlight in edge computing environments for improved performance and reduced latency.
- **[Alerts](10-alerts.md)**: Learn how to set up alerts to notify you via text (SMS) or email when specific conditions are met in your visual applications.


By exploring these resources and sample applications, you'll be well on your way to building powerful visual applications using Groundlight's computer vision and natural language capabilities.
4 changes: 2 additions & 2 deletions docs/docs/other-ways-to-use/1-stream-processor.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ It supports a variety of input sources, including:
- Image directories
- Image URLs

The Stream Processor can be combined with [Groundlight Alerts](../guide/9-alerts.md) to create a simple video analytics system. For example, you could use the Stream Processor to process a video stream from a security camera and send an alert when a person is detected.
The Stream Processor can be combined with [Groundlight Alerts](../guide/10-alerts.md) to create a simple video analytics system. For example, you could use the Stream Processor to process a video stream from a security camera and send an alert when a person is detected.

## Prerequisites:
You will need:
Expand Down Expand Up @@ -79,4 +79,4 @@ The Groundlight Stream Processor is lightweight and can be run on a Raspberry Pi
## Combining with Groundlight Alerts
The Stream Processor submits frames to Groundlight, but it does not do anything with the results.

In order to build a useful alerting system, you can combine the Stream Processor with [Groundlight Alerts](../guide/9-alerts.md).
In order to build a useful alerting system, you can combine the Stream Processor with [Groundlight Alerts](../guide/10-alerts.md).
Loading