Skip to content

Latest commit

 

History

History
151 lines (111 loc) · 3.7 KB

BinaryAccuracy.md

File metadata and controls

151 lines (111 loc) · 3.7 KB
Raw

TFSimilarity.classification_metrics.BinaryAccuracy

Calculates how often the query label matches the derived lookup label.

Inherits From: ClassificationMetric, ABC

TFSimilarity.classification_metrics.BinaryAccuracy(
    name: str = binary_accuracy
) -> None

Accuracy is technically (TP+TN)/(TP+FP+TN+FN), but here we filter all queries above the distance threshold. In the case of binary matching, this makes all the TPs and FPs below the distance threshold and all the TNs and FNs above the distance threshold.

As we are only concerned with the matches below the distance threshold, the accuracy simplifies to TP/(TP+FP) and is equivalent to the precision with respect to the unfiltered queries. However, we also want to consider the query coverage at the distance threshold, i.e., the percentage of queries that retrun a match, computed as (TP+FP)/(TP+FP+TN+FN). Therefore, we can take $ precision imes query_coverage $ to produce a measure that capture the precision scaled by the query coverage. This simplifies down to the binary accuracy presented here, giving TP/(TP+FP+TN+FN).

args: name: Name associated with a specific metric object, e.g., [email protected]

Usage with tf.similarity.models.SimilarityModel():

model.calibrate(x=query_examples,
                y=query_labels,
                calibration_metric='binary_accuracy')

Methods

compute

View source

compute(
    tp: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    fp: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    tn: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    fn: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    count: int
) -> <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>

Compute the classification metric.

The compute() method supports computing the metric for a set of values, where each value represents the counts at a specific distance threshold.

Args
tp A 1D FloatTensor containing the count of True Positives at each distance threshold.
fp A 1D FloatTensor containing the count of False Positives at each distance threshold.
tn A 1D FloatTensor containing the count of True Negatives at each distance threshold.
fn A 1D FloatTensor containing the count of False Negatives at each distance threshold.
count The total number of queries
Returns
A 1D FloatTensor containing the metric at each distance threshold.

get_config

View source

get_config()