add an example to IO docs

[baggepinnen]

No description provided.

[cstjean]

Thank you for the example! I'll confess that I'm still confused. I came here looking for solutions to #2905 (i.e. general black box u = some_arbitrary_f(current_state) control). From my understanding, https://docs.sciml.ai/ModelingToolkit/dev/basics/Events/#Discrete-events-support (and https://docs.sciml.ai/DiffEqDocs/stable/features/callback_functions/#Example-2:-A-Control-Problem) looks like the best solution out there. I was trying my best to see if generate_control_function would be useful, and my rough impression from reading the docs it's that it's kinda meant for "continuous control" (classical control theory problems, I guess?) This isn't what I want, I think, but it's still interesting. However, while I see that you can pass an u to f above, and it returns "the dynamics of the system", how would you use that to achieve control?

I think I'm confused about the generate_control_function name, and it has a technical meaning which eludes me. Googling for "control function" didn't yield anything useful.

[baggepinnen]

It sounds to me like your u = some_arbitrary_f(current_state) is a state feedback controller, is there a reason you cannot implement this as an equation in your model? Like here
https://help.juliahub.com/juliasimcontrol/dev/examples/mtk_control/#State-feedback-using-ModelingToolkit

This isn't what I want, I think, but it's still interesting.

What exactly is it that you want?

[cstjean]

Like here
https://help.juliahub.com/juliasimcontrol/dev/examples/mtk_control/#State-feedback-using-ModelingToolkit

My background is reinforcement learning, I struggle to read through the documentation on control topics. I think I'm just out of my depth here, so I'll refrain from further comments. Thank you for the link.

What exactly is it that you want?

In plain Julia, it'd be something like:

control_setpoint(x) = maximum(u->Q_value(x, u), [10,15,20,25])  # choose the setpoint which has highest Q value

integral = x = 0.0
for t in 0:10
	u = control_setpoint(x)   # assumed to be constant for that timestep
	x = g(x, u, t)       # arbitrary non-linear state dynamic
	integral += some_f(x)     # Riemann sum
end

# learn a Q_value function to maximize `integral`

I've been looking for how to express the u = control_setpoint(x) line in ModelingToolkit. Discrete events look like a valid way to do it.

I don't want to derail this further, thank you again for providing an example in this PR.

[baggepinnen]

Your example is completely in discrete time, including the dynamics, while generate_control_function is for continuous time systems.

Hybrid continuous-discrete systems are not yet fully supported in MTK (as described here), so you cannot implement a discrete-time controller in any convenient way yet.

Furthermore, MTK will not give any guarantees of what x actually is, it can change between versions of MTK. You thus always have to implement feedback (u = control_setpoint(x)) in terms of variables you specify yourself rather than the state.

[cstjean]

In my real example, the state is continuous and changes quickly (minutes), while my setpoints (my control signal u) ought to stay put for hours (i.e. could be a ConstantInterpolation(u,t)). If I use discrete events to change the setpoints, and I specify those setpoints as d_discontinuities, won't it work?

[ChrisRackauckas]

@thazhemadam how could the formatter fail after taking its suggestions? Different versions? 😅

[github-actions]

[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

[github-actions]

[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

[github-actions]

[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

[github-actions]

[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

[github-actions]

[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

[ChrisRackauckas]

😅 review dog, woof.