Add utility to ensure that a kernel with arbitrary workers exist.

[aadcg]

A follow-up to lmj#44 since the maintainer didn't reply.

CC @Ambrevar.

Thanks.

[Ambrevar]

Problem: when you replace an existing kernel, you don't kill the old one, thus leaving it hanging.
Either you should call end-kernel, or you should not replace the existing kernel.

[dkochmanski]

I think that calling make-kernel when another kernel exists should signal a condition with appropriate restarts (i.e reuse-existing-kernel [with appropriate test to check whether the count matches], end-kernel-and-create-new-kernel, something else (?)).

(just chipping in with ideas)

[swapneils]

I think that calling make-kernel when another kernel exists should signal a condition with appropriate restarts (i.e reuse-existing-kernel [with appropriate test to check whether the count matches], end-kernel-and-create-new-kernel, something else (?)).

(just chipping in with ideas)

Wouldn't this interfere with using multiple kernels?

I feel like that's an important ability to retain. For instance, to allow restricting the number of created threads and occasionally creating new ones when a certain set of tasks absolutely requires the separate threads. Or when a set of tasks will be running long enough to handle them separately from whatever the global kernel is being used for.

Problem: when you replace an existing kernel, you don't kill the old one, thus leaving it hanging. Either you should call end-kernel, or you should not replace the existing kernel.

Building off of this, I'd recommend a specific key argument for "replace", which if true would end and replace the previous kernel and if nil would just return the created kernel.
We also need a "wait" key argument to pass to end-kernel, since systems which close threads slowly might cause issues for either :wait nil (unexpected crashing, intuitively, but I haven't checked) or :wait t (unnecessarily delaying computation) depending on context.

For maximal modularity (in this case, not influencing the current kernel object unless necessary), "replace" should ideally be default nil, but the existence of the argument would inform people they need to set it if they're ensuring the global kernel value.

Incidentally, it would also be nice to use multiple return values, the second indicating whether a valid prior kernel exists (t for "valid", nil for "new created"), and the third providing the shutdown manager for the end-kernel command, if one was produced. Something like:

(defun ensure-kernel (worker-count &rest other-keys &key replace wait &allow-other-keys)
  "Return `*kernel*', a kernel instance featuring WORKER-COUNT.
When `replace' is true (non-nil), the current environment's *kernel* is ended and replaced (if invalid). Otherwise, this function merely returns a valid kernel.
The `wait' argument is passed to the `end-kernel' function if replacing the current *kernel*"
  ;; Notice that when *kernel* exists with 2 workers, then calling:
  ;; (ensure-kernel 2 :name "a-different-name-from-the-one-set-in-*kernel*")
  ;; has no effect since the number of workers matches.
  (if (and *kernel*
           (eq (length (workers *kernel*)) worker-count))
      (values *kernel* t nil)
      (let ((new-kernel (apply #'make-kernel worker-count other-keys))
            (shutdown-manager nil))
        (when replace 
          (setf shutdown-manager (end-kernel :wait wait))
          (setf *kernel* new-kernel))
        (values new-kernel nil shutdown-manager))))

Separately from the above: Is there a benefit to exactly specifying the kernel count? Or should we just use >= instead of eq, to ensure at least this worker-count workers?