Skip to content

Commit b924f58

Browse files
committed
Incorporate inputQuota, measureInputUsage(), and quota exceeded errors
Follows webmachinelearning/writing-assistance-apis#43.
1 parent 5e7cd44 commit b924f58

File tree

2 files changed

+220
-21
lines changed

2 files changed

+220
-21
lines changed

README.md

+64
Original file line numberDiff line numberDiff line change
@@ -175,6 +175,70 @@ It is also nicely future-extensible by adding more events and properties to the
175175
Finally, note that there is a sort of precedent in the (never-shipped) [`FetchObserver` design](https://github.com/whatwg/fetch/issues/447#issuecomment-281731850).
176176
</details>
177177

178+
### Too-large inputs
179+
180+
It's possible that the inputs given for translation or language detection might be too large for the underlying machine learning model to handle. Although there are often techniques that allow implementations to break up the inputs into smaller chunks, and combine the results, the APIs have some facilities to allow browsers to signal such too-large inputs.
181+
182+
Whenever any API call fails due to too-large input, it is rejected with a `QuotaExceededError`. This is a proposed new type of exception, which subclasses `DOMException`, and replaces the web platform's existing `"QuotaExceededError"` `DOMException`. See [whatwg/webidl#1465](https://github.com/whatwg/webidl/pull/1465) for this proposal. For our purposes, the important part is that it has the following properties:
183+
184+
* `requested`: how much "usage" the input consists of
185+
* `quota`: how much "usage" was available (which will be less than `requested`)
186+
187+
The "usage" concept is specific to the implementation, and could be something like string length, or [language model tokens](https://arxiv.org/abs/2404.08335).
188+
189+
This allows detecting failures due to overlarge inputs and giving clear feedback to the user, with code such as the following:
190+
191+
```js
192+
const detector = await ai.languageDetector.create();
193+
194+
try {
195+
console.log(await detector.detect(potentiallyLargeInput));
196+
} catch (e) {
197+
if (e.name === "QuotaExceededError") {
198+
console.error(`Input too large! You tried to detect the language of ${e.requested} tokens, but ${e.quota} is the max supported.`);
199+
200+
// Or maybe:
201+
console.error(`Input too large! It's ${e.requested / e.quota}x as large as the maximum possible input size.`);
202+
}
203+
}
204+
```
205+
206+
In some cases, instead of providing errors after the fact, the developer needs to be able to communicate to the user how close they are to the limit. For this, they can use the `inputQuota` property and the `measureInputUsage()` method on the translator or language detector objects:
207+
208+
```js
209+
const translator = await ai.translator.create({
210+
sourceLanguage: "en",
211+
targetLanguage: "jp"
212+
});
213+
meterEl.max = translator.inputQuota;
214+
215+
textbox.addEventListener("input", () => {
216+
meterEl.value = await translator.measureInputUsage(textbox.value);
217+
submitButton.disabled = meterEl.value > meterEl.max;
218+
});
219+
220+
submitButton.addEventListener("click", () => {
221+
console.log(translator.translate(textbox.value));
222+
});
223+
```
224+
225+
Note that if an implementation does not have any limits, e.g. because it uses techniques to split up the input and process it a bit at a time, then `inputQuota` will be `+Infinity` and `measureInputUsage()` will always return 0.
226+
227+
Developers need to be cautious not to over-use this API, however, as it requires a round-trip to the underlying model. That is, the following code is bad, as it performs two round trips with the same input:
228+
229+
```js
230+
// DO NOT DO THIS
231+
232+
const usage = await translator.measureInputUsage(input);
233+
if (usage < translator.inputQuota) {
234+
console.log(await translator.translate(input));
235+
} else {
236+
console.error(`Input too large!`);
237+
}
238+
```
239+
240+
If you're planning to call `translate()` anyway, then using a pattern like the one that opened this section, which catches `QuotaExceededError`s, is more efficient than using `measureInputUsage()` plus a conditional call to `translate()`.
241+
178242
### Destruction and aborting
179243

180244
The API comes equipped with a couple of `signal` options that accept `AbortSignal`s, to allow aborting the creation of the translator/language detector, or the translation/language detection operations themselves:

0 commit comments

Comments
 (0)