Is there a way to set the task priority?

[FlyHighest]

I'm developing a server that handles time-consuming tasks(ai generative model inference). I want some tasks to execute with high priority. Is there a way to set the priority in the queue?

[kemingy]

Nope. I used to implement this feature but I feel it's not suitable. It should be done by the gateway or service mesh.

If you have multiple replicas for both high-priority and low-priority tasks, the ingress controller should be responsible to dispatch the traffic according to the load and priority. If you only have one replica, maybe you need to control the client side.

[VoVAllen]

@kemingy Do you think we should implement the pull-mode for mosec, so that it can pull the requests from a message queue?

[kemingy]

I think it can be a feature. Then the results need to be pushed to another message queue.

[gaocegege]

Personally, I prefer the push-based model. A separate agent watches the message queue and sends a req to mosec. Then it writes the resp to another one.

Then mosec could keep simple.