downloadState requires server restart

[bones-was-here]

Background information

• Dendrite version or git SHA: 0.13.3
• SQLite3 or Postgres?: postgres
• Running in Docker?: no
• go version: 1.21
• Client used (if applicable): element web

Description

• What is the problem: downloadState often hangs indefinitely unless dendrite is first restarted
• Who is affected: admins trying to fix room state issues
• How is this bug manifesting: curl doesn't exit until killed and doesn't receive any HTTP BODY, nothing is logged (except for request cancelled when curl is killed)
• When did this first appear: unknown, but it's quite old

Steps to reproduce

• Unsure how to get the cache into a state where this will happen, but it seems more likely than not when doing downloadState

For a long time I didn't know why downloadState sometimes worked and other times just hung.
Is it possible that cache<>db synchronisation issues affect other aspects of server operation too? Related: #3190