Inference failure in callback loop

[charleskawczynski]

This happens in a lot of cases in https://buildkite.com/clima/climaatmos-ci/builds/20627#01920570-27e2-49a1-8b67-bc156f98d825

JET-test failed at /central/scratch/esm/slurm-buildkite/climaatmos-ci/20627/climaatmos-ci/perf/jet_report_nfailures.jl:54
--
  | Expression: #= /central/scratch/esm/slurm-buildkite/climaatmos-ci/20627/climaatmos-ci/perf/jet_report_nfailures.jl:54 =# JET.@test_opt ignored_modules = (NCDatasets, CUDA, HDF5, LLVM, GPUCompiler) SciMLBase.step!(integrator)
  | ═════ 1 possible error found ═════
  | ┌ step!(integrator::ClimaTimeSteppers.DistributedODEIntegrator{…}) @ ClimaTimeSteppers /central/scratch/esm/slurm-buildkite/climaatmos-ci/depot/default/packages/ClimaTimeSteppers/Na6yl/src/integrators.jl:203
  | │┌ __step!(integrator::ClimaTimeSteppers.DistributedODEIntegrator{…}) @ ClimaTimeSteppers /central/scratch/esm/slurm-buildkite/climaatmos-ci/depot/default/packages/ClimaTimeSteppers/Na6yl/src/integrators.jl:250
  | ││ runtime dispatch detected: %140::Any(%141::ClimaCore.Fields.FieldVector{Float64, @NamedTuple{c::ClimaCore.Fields.Field{…}, f::ClimaCore.Fields.Field{…}}}, %142::Float64, %133::ClimaTimeSteppers.DistributedODEIntegrator{…})::Any
  | │└────────────────────

I think we should add some JET tests for this. Maybe UnrolledUtilities is a good candidate to fix the JET failures here? We might even be able to use Base.Cartesian.@nif

Also, just to note, perhaps we need to remove the NVTX.@range to fix this.

[charleskawczynski]

Closed by #318