|
| 1 | +# Detector Answer Modalities |
| 2 | + |
| 3 | +Groundlight supports a variety of answer modalities. Thus far, all of the examples we have provided are for binary classification detectors. However, Groundlight also supports counting, multi-class, and object detection, text detectors. |
| 4 | + |
| 5 | +## Counting Detectors |
| 6 | + |
| 7 | +Counting detectors are used to count the number of objects in an image. Groundlight's counting detectors also return bounding boxes around the objects they count. |
| 8 | + |
| 9 | +```python notest |
| 10 | +from groundlight import ExperimentalApi |
| 11 | +gl_exp = ExperimentalApi() |
| 12 | + |
| 13 | +# highlight-start |
| 14 | +detector = gl_exp.create_counting_detector( |
| 15 | + name="car-counter", |
| 16 | + query="How many cars are in the parking lot?", |
| 17 | + max_count=20, |
| 18 | + confidence_threshold=0.2, |
| 19 | +) |
| 20 | +# highlight-end |
| 21 | +``` |
| 22 | + |
| 23 | +Counting detectors should be provided with a query that asks "how many" objects are in the image. |
| 24 | + |
| 25 | +A maximum count (of 50 or fewer) must be specified when creating a counting detector. This is the maximum number of objects that the detector will count in an image. Groundlight's ML models are optimized for counting up to 20 objects, but you can increase the maximum count to 50 if needed. If you have an application that requires counting more than 50 objects, please [contact us ](mailto:[email protected]). |
| 26 | + |
| 27 | +:::note |
| 28 | +Counting Detectors are available on [Business and Enterprise plans](https://www.groundlight.ai/pricing). |
| 29 | +::: |
| 30 | + |
| 31 | +### Submit an Image Query to a Counting Detector |
| 32 | + |
| 33 | +Now that you have created a counting detector, you can submit an image query to it. |
| 34 | + |
| 35 | +```python notest |
| 36 | +from groundlight import ExperimentalApi |
| 37 | +gl_exp = ExperimentalApi() |
| 38 | + |
| 39 | +# highlight-start |
| 40 | +# Use the detector from the previous example to count the number of cars in an image |
| 41 | +image_query = gl_exp.submit_image_query(detector, "path/to/image.jpg") |
| 42 | +# highlight-end |
| 43 | + |
| 44 | +print(f"Counted {image_query.result.count} cars") |
| 45 | +print(f"Confidence: {image_query.result.confidence}") |
| 46 | +print(f"Bounding Boxes: {image_query.rois}") |
| 47 | +``` |
| 48 | + |
| 49 | +In the case of counting detectors, the `count` attribute of the result object will contain the number of objects counted in the image. The `confidence` attribute represents the confidence level in the specific count. Note that this implies that confidences may be lower (on average) for counting detectors with a higher maximum count. |
| 50 | + |
| 51 | +<!-- TODO: display an example image with bounding boxes --> |
| 52 | + |
| 53 | +:::tip Drawing Bounding Boxes |
| 54 | +You can visualize the bounding boxes returned by counting detectors using a library like OpenCV. Here's an example of how to draw bounding boxes on an image: |
| 55 | + |
| 56 | +```python notest |
| 57 | +import cv2 |
| 58 | +import numpy as np |
| 59 | + |
| 60 | +def draw_bounding_boxes(image_path, rois): |
| 61 | + """ |
| 62 | + Draw bounding boxes on an image based on ROIs returned from a counting detector. |
| 63 | +
|
| 64 | + Args: |
| 65 | + image_path: Path to the image file |
| 66 | + rois: List of ROI objects returned from image_query.rois |
| 67 | + """ |
| 68 | + image = cv2.imread(image_path) |
| 69 | + if image is None: |
| 70 | + raise ValueError(f"Could not read image from {image_path}") |
| 71 | + height, width = image.shape[:2] |
| 72 | + |
| 73 | + # Draw bounding boxes |
| 74 | + for roi in rois: |
| 75 | + x1 = int(roi.geometry.left * width) |
| 76 | + y1 = int(roi.geometry.top * height) |
| 77 | + x2 = int(roi.geometry.right * width) |
| 78 | + y2 = int(roi.geometry.bottom * height) |
| 79 | + cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2) |
| 80 | + label_text = f"{roi.label}: {roi.score:.2f}" |
| 81 | + cv2.putText(image, label_text, (x1, y1-10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2) |
| 82 | + |
| 83 | + # Display the image |
| 84 | + cv2.imshow("Image with Bounding Boxes", image) |
| 85 | + cv2.waitKey(0) |
| 86 | + cv2.destroyAllWindows() |
| 87 | + |
| 88 | +# Example usage: |
| 89 | +# image_query = gl.submit_image_query(detector, "path/to/image.jpg") |
| 90 | +# draw_bounding_boxes("path/to/image.jpg", image_query.rois) |
| 91 | +``` |
| 92 | +::: |
| 93 | + |
| 94 | +### Add a label to a Counting Detector |
| 95 | + |
| 96 | +The Groundlight API allows you to add labels to image queries, including Region of Interest (ROI) data. |
| 97 | +When adding a label to a counting detector, if you include ROIs, the number of ROIs should match |
| 98 | +the count you are labeling. |
| 99 | + |
| 100 | +```python notest |
| 101 | +from groundlight import ExperimentalApi |
| 102 | +gl_exp = ExperimentalApi() |
| 103 | + |
| 104 | +# highlight-start |
| 105 | +# Add a count label with corresponding ROIs to an image query |
| 106 | +roi1 = gl_exp.create_roi("car", (0.1, 0.2), (0.2, 0.3)) |
| 107 | +roi2 = gl_exp.create_roi("car", (0.4, 0.4), (0.5, 0.6)) |
| 108 | +roi3 = gl_exp.create_roi("car", (0.6, 0.5), (0.8, 0.9)) |
| 109 | +rois = [roi1, roi2, roi3] |
| 110 | +gl_exp.add_label(image_query, label=len(rois), rois=rois) |
| 111 | +# highlight-end |
| 112 | +``` |
| 113 | + |
| 114 | +## [BETA] Multi-Class Detectors |
| 115 | + |
| 116 | +If you want to classify images into multiple categories, you can create a multi-class detector. |
| 117 | + |
| 118 | +```python notest |
| 119 | +from groundlight import ExperimentalApi |
| 120 | +gl_exp = ExperimentalApi() |
| 121 | + |
| 122 | +# highlight-start |
| 123 | +class_names = ["Golden Retriever", "Labrador Retriever", "German Shepherd", "Other"] |
| 124 | +detector = gl_exp.create_multiclass_detector( |
| 125 | + name, query="What kind of dog is this?", |
| 126 | + class_names=class_names, |
| 127 | +) |
| 128 | +# highlight-end |
| 129 | +``` |
| 130 | + |
| 131 | +:::tip |
| 132 | +We recommend adding an "Other" class to your multi-class detector to handle cases where the image does not belong to any of the pre-defined classes. |
| 133 | +::: |
| 134 | + |
| 135 | +### Submit an Image Query to a Multi-Class Detector |
| 136 | + |
| 137 | +Now that you have created a multi-class detector, you can submit an image query to it. |
| 138 | + |
| 139 | +```python notest |
| 140 | +from groundlight import ExperimentalApi |
| 141 | +gl_exp = ExperimentalApi() |
| 142 | + |
| 143 | +# highlight-start |
| 144 | +# Use the detector from the previous example to classify the breed of a dog in an image |
| 145 | +image_query = gl_exp.submit_image_query(detector, "path/to/image.jpg") |
| 146 | +# highlight-end |
| 147 | + |
| 148 | +print(f"Result: {image_query.result.label}") |
| 149 | +print(f"Confidence: {image_query.result.confidence}") |
| 150 | +``` |
| 151 | + |
| 152 | +Multi-class detectors return a `label` attribute in the result object, which contains the predicted class label. The `label` attribute will be one of the class names provided when creating the detector. The `confidence` attribute represents the confidence level in the predicted class, which is a value between `1/len(class_names)` and 1. |
| 153 | + |
| 154 | +### Add a label to a Multi-Class Detector |
| 155 | + |
| 156 | +To provide ground truth labels for multi-class detectors, you can specify the label of the correct class. |
| 157 | + |
| 158 | +```python notest |
| 159 | +from groundlight import ExperimentalApi |
| 160 | +gl_exp = ExperimentalApi() |
| 161 | + |
| 162 | +# highlight-start |
| 163 | +# Add a multi-class label to an image query |
| 164 | +gl_exp.add_label(image_query, label="German Shepherd") |
| 165 | +# highlight-end |
| 166 | +``` |
| 167 | + |
| 168 | +<!-- TODO: text, object detection modes --> |
0 commit comments