Implement symmetric versions of models

[pbrehmer]

In this PR we add symmetric model Hamiltonians using MPSKitModels.jl. This allows us to run, e.g., a $\mathrm{U}(1)$ symmetric optimization of the Heisenberg model.

[codecov]

Codecov Report

Attention: Patch coverage is 87.80488% with 10 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/operators/models.jl	75.00%	9 Missing ⚠️
src/utility/symmetrization.jl	94.73%	1 Missing ⚠️

Files with missing lines	Coverage Δ
src/PEPSKit.jl	100.00% <ø> (ø)
src/algorithms/peps_opt.jl	95.45% <100.00%> (+0.21%)	⬆️
src/algorithms/toolbox.jl	100.00% <100.00%> (ø)
src/operators/lattices/squarelattice.jl	100.00% <100.00%> (ø)
src/operators/localoperator.jl	54.00% <ø> (-20.58%)	⬇️
src/utility/symmetrization.jl	84.11% <94.73%> (+3.26%)	⬆️
src/operators/models.jl	80.00% <75.00%> (-20.00%)	⬇️

[pbrehmer]

I noticed a problem with the symmetrized optimization when I looked at Lander's symmetrization code I hadn't taken into account before: already at the retract stage the PEPS needs to be symmetrized in order for the line search to work properly. Before, I only symmetrized at the finalize! stage which also kind of worked but leads to problems, I think.

Of course it is not very nice to have retract, inner and finalize! as keyword arguments in fixedpoint but since we anyway want to change the optimizer, I kept it like that for now for correctness. Then we at least know what the new optimization package should be capable of (i.e. custom retraction/finalization routines or something similar).

[pbrehmer]

The models now overload/mimic the MPSKitModels definitions and I defined a InfiniteSquare subtype of AbstractLattice{2}. I'm not quite sure how to best handle the defaults and overwriting definitions from MPSKitModels (there are some warnings during compilation). But other than that, the PR should be mergable :)

[lkdvos]

I think the warnings are because you defined the lattice to default to the peps models, while they already had default values to be the infinitechain. Probably it's better to just not give them a default value here.
Considering the symmetrization, if we add these keywords to the fixedpoint routine than that routine is really just becoming more and more of a carbon copy of the optimization calls. Maybe we can add a symmetrization kwarg instead that automatically inserts the correct retraction etc?

[pbrehmer]

I think the warnings are because you defined the lattice to default to the peps models, while they already had default values to be the infinitechain. Probably it's better to just not give them a default value here.

Whoops, I didn't have that in mind but I'm also okay with having to specify InfiniteSquare() when you initialize the Hamiltonian. I just thought that would make the user interface again a bit more complicated and convoluted but if we want to be compatible with MPSKitModels then that's probably the best compromise.

Maybe we can add a symmetrization kwarg instead that automatically inserts the correct retraction etc?

The problem with just having the symmetrization kwarg is that we still need a callback/finalize kwarg (for example if one wants to implement some kind of checkpoint saving mechanism). I do of course agree that fixedpoint is now just converging to a carbon copy of the optimization routine but it is still more convenient to have some wrapper around the optimize call.

At this point the optimization interface with all the arguments has grown too much and I think we should consider doing a clean rewrite. Also if we switch to Optim.jl/Manopt.jl then things will again change a bit (i.e. how to implement callbacks and the symmetrization). So I'm not sure what the best way is to proceed in the meantime. I could implement a symmetrization kwarg which accepts nothing or SymmetrizationStyle and then inserts the correct arguments but I would still need the finalize! kwarg. What do you think?

[lkdvos]

I think it's definitely nicer to have a symmetrize and finalize kwarg in that case. I think you could wrap the user-provided finalizer by first doing the symmetrization and then the finalizer, and at least this is hiding the fact that you need to know about retractions and inner products etc that have to be compatible.

My general feeling is a bit like this: either we try and do things automatically, in which case it should be possible to use the interface without knowing too much about the internals, or we consider this to be an expert usage case. If the former, I think symmetrize is a bit more intuitive, and if the latter, it's not actually that hard to hijack the optimization routine yourself to hack in custom finalizers etc. I don't like mixing up these two concepts, as that typically ends up with being too elaborate for casual use, while not being flexible if you want to really fiddle with it

[pbrehmer]

I think it's definitely nicer to have a symmetrize and finalize kwarg in that case.

I would also think so, I'll get to it after the weekend!

My general feeling is a bit like this: either we try and do things automatically, in which case it should be possible to use the interface without knowing too much about the internals, or we consider this to be an expert usage case.

I definitely agree with the general idea. I feel like that we already are somewhat in the middle ground between an expert and automatic user interface, so that's why I would like to redesign a bit at some point (especially the way all the parameters are set). But for now I will just implement the symmetrization kwarg.