Add warmup client CLI option

[vrdn-23]

What does this PR do?

This allows for a CLI option to enable warmup requests for models before server start-up. Currently serving the first request in, takes more than a second for most models (see the issue shown below).

Fixes #580

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@alvarobartt @Narsil @kozistr

[vrdn-23]

Disclaimer: I have never done any coding in Rust before, so please do let me know if I am doing something blatantly wrong. I am merely following the pattern followed with the auto_truncate option and using that as a basis to figure out which all places to define warmup-model. I've compiled this locally without failures and tested it and it seems to be working fine. I will also see if I can test this on a CUDA device next

[kozistr]

@vrdn-23 hi! As you mentioned, it looks like only models with Flash Attention (== unpadded model) warm up explicitly through warmup() at the beginning of the server code.

I also found that health() does a similar thing for all model inference with a length of 1 sequence at the start of the server (right before the warmup). And it might not be enough for warming up, it's for advanced health check purposes.

I'm not sure yet which is the best practice for warming up with Candle, but I also believe that enabling warmup for all models seems beneficial. Maybe, we could just remove line 251 and warmup essentially. (However, to do this, I think we need to adjust the warmup() parameters so that the warmup doesn't take too long.)

Looking forward to maintainers' feedback :)

@alvarobartt @Narsil

[Narsil]

I think we should just warmup by default all the time given the times we're reading.

In general, flash models should be the standard nowadays (the lack of padding makes them generally much faster), but regardless warming up is OK for any model.

I think what you are experiencing is probably something loading/up, some kernel JIT or something like that.

And looking at the warmup code, it definitely seems to not be considering well the non padded models, but I think it's fine for now (if it doesn't crash)

[Narsil]

#592

[vrdn-23]

Thanks for the review @Narsil @kozistr ! I'll close this in favor of the new PR @Narsil has raised