Custom tokenizer not actually possible?

[patricklucas]

Both the documentation and TypeScript types suggest you can pass a custom tokenizer as a function, but as far as I can tell this actually does nothing. Looking at the code, the tokenize option is only ever used in a switch statement that looks for the known string values, and no code path ever tries to call it as a function (the default branch of the switch assumes "strict"): https://github.com/nextapps-de/flexsearch/blob/961c3ae84a87fb3af2a52047fd7c8adc8949b86d/src/index.js#L166C33-L166C39

Can anyone else confirm this?