Gauge fixing for nontrivial symmetries and unit cells

[leburgel]

I was trying out the new updates on a model with $\mathbb{Z}_2$ symmetry and a $2 \times 2$ unit cell as a test, and I ran into some issues with the gauge fixing.

Unit cells should be fine in principle, but in the current implementation this can give some space mismatches when the edge tensor spaces aren't the same for every site, I think due to this line
https://github.com/quantumghent/PEPSKit.jl/blob/a24ac36321dfbccb0e1e12b3eb9378411f0a4ab5/src/algorithms/ctmrg.jl#L95
not taking into account the site dependence, which is easily fixed.

Symmetries are bit more tricky, since there's two steps where the block structure isn't taken into account,
https://github.com/quantumghent/PEPSKit.jl/blob/a24ac36321dfbccb0e1e12b3eb9378411f0a4ab5/src/algorithms/ctmrg.jl#L82-L83
and
https://github.com/quantumghent/PEPSKit.jl/blob/a24ac36321dfbccb0e1e12b3eb9378411f0a4ab5/src/algorithms/ctmrg.jl#L93-L95
For the first one, projecting out one specific configuration of physical indices isn't very natural in the presence of symmetries. We could just do it, but I'm not sure if this is then still what we want. The second one seems more involved, mainly because I'm not sure what's happening exactly. Just trying the closest block-diagonal analogue to what's there breaks the $\mathbb{Z}_2$ version of the test_gauge_fixing.jl test, so I didn't manage to patch it without understanding it.

Probably the case of symmetries just requires the use of a different gauge-fixing procedures, but just wanted to point this out in case there's a quick fix after all. Any thoughts @pbrehmer, @lkdvos?

[leburgel]

Ok, after some more messing around I gave up on not thinking and actually implemented the "general" gauge fixing algorithm from the paper. This works for $\mathbb{Z}_2$ and unit cells, but it breaks the actual AD optimization example since KrylovKit.eigsolve is not differentiable (yet). I think Lukas has an rrule for that somewhere, so I'll see if I can get the whole thing to work after I get my hands on that.

[pbrehmer]

Thanks for following up on that! I also had an implementation lying around for the "general" gauge fixing algorithm which didn't AD, but your implementation using MPSKit routines is much nicer. So this is finally a good time to get it working. Great to hear that this works with unit cells and symmetries (I was struggling with unit cells previously). Maybe we can discuss the details next week.

[leburgel]

Closed by #19.