Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing handling of terminal states for POMDPs #603

Open
AlexBork opened this issue Aug 28, 2024 · 2 comments · May be fixed by #617
Open

Missing handling of terminal states for POMDPs #603

AlexBork opened this issue Aug 28, 2024 · 2 comments · May be fixed by #617

Comments

@AlexBork
Copy link
Contributor

States that are marked as terminal while building a POMDP model (for example due to violating the left-hand side of an Until) are not handled appropriately. By default, for terminal states, only one self-loop without action label is added. This, however, may lead to violation of the requirement that states with the same observation must have the same actions enabled.

This can be worked around with by using option --buildfull, but terminal states should be handled properly to reduce model building times.

This issue is based on the discussion on issue #48.

@sjunges
Copy link
Contributor

sjunges commented Aug 28, 2024

It would probably be easy to allow this in makeCanonic and add the additional self-loops there?

@AlexBork
Copy link
Contributor Author

I'm not sure if it's not a bit convoluted to get the information to makeCanonic. We need to be careful to distinguish between problems that are actual modelling errors (with respect to the POMDP requirements) and ones that arise due to our handling.

I think the most robust solution would be to handle the absorbing/terminal states correctly in the state generation, i.e. adding the self-loop transitions instead of nothing or transitions to successor states. I have implemented this yesterday and it seems to run fine. It should also introduce only minimal overhead.

@AlexBork AlexBork linked a pull request Sep 5, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants