Using whisper in a webservice

[mp1smw]

I am wanting to build this into a web service, which there are examples of but I am wanting to know if there are any elements of the chain below that can be made static and shared across all threads on the web server. So the question is for each call do I have to make all of the calls below or can the factory and the builder be made only once and share across service calls. I do not know enough about how the underlying code works to know if this is a dumb question so would be good to get some thoughts from someone who knows?

var whisperFactory = WhisperFactory.FromPath(modelFileName);

var builder = whisperFactory.CreateBuilder()
.WithProbabilities()
.WithLanguage("en");

processor = builder.Build();

processor.ProcessAsync(samples.AsMemory(0, currentSampleIndex));

[sandrohanea]

Hello @mp1smw ,

WhisperFactory should be thread safe and can be used from a singleton in your Web Service (and shared across all threads).

However, each request should create their own processor.

The builder is a lightweight object as well that hosts only the options, so you can either create it for every request (if the options are different) or keep it in a singleton as well if you are using the same options all the time.

This way, you'll ensure that the model will be loaded in memory only once for the entire application and each thread can use their own processor.

I hope it makes sense.

[mp1smw]

That makes perfect sense and exactly what I was hoping for so thanks very much for the swift and concise reply.