Roadmap

[lkdvos]

In an attempt to make a more concrete plan for the near future, here are the results of the discussions on current features, known issues, planned features and release schedule.

v0.5

General consolidation of ongoing PRs, with particular care to be specific about breaking changes, fixing public API and general usability improvements.

• Add bond truncation algorithms for time evolution #128 seems ready and can be merged as is
• Implement keyword selector #130 should be finalized to improve the configuration of hyper parameters.
• We want to identify a set of examples outside of our test cases that have less focus on stability, in order to serve as test beds for tweaking settings and improving robustness
• We want to set up (and possibly start filling in) the infrastructure for more documentation features, to generally improve the code usability. In particular, this should include notebooks with a make script, references, and a generic outline of the docs pages.
• We want to start keeping a changelog

v0.6

This version should focus on resolving long-standing stability issues, either by improving on them or expanding explanations about how to deal with common problems through the documentation system.

Gradient fixed point equation:

There are several issues here that require addressing. In order to solve the f(x) = x reverse pass with a linear solver, we have the following options:

• GMRES: can plateau and simply not converge, need unreasonable Krylovdimensions
• BICGSTAB: sometimes works, sometimes doesn't. Current default?
• Arnoldi: seems to be the most stable, given that KrylovKit is made less strict. Probably this should be the next default.

We should clearly document the options, possible issues and solutions, pros and cons, ... in the docs and identify the most stable option as the default.

Optimization initialization stage

At the start of an optimization run, it seems like the stability is the most fragile.
We try to identify several improvements to alleviate this:

• We probably want to relax the requirements for converging subalgorithms in the beginning. This should push us past this initial stage quicker, as these unoptimized states tend to be pathological.
• In order to allow for relaxed tolerances, we need a more lenient linesearch that actually allows this.
• One approach to have this is the inclusion of algorithms with tolerances that are scaled based on the current gradient norm. Reference implementation in MPSKit we can reuse.
• We should outline a multi-stage approach to optimizing in the docs, where we attempt to explain that the first stage is centered around getting a good initial state, with larger and more lenient steps, while the second stage is centered around more accurate convergence through stricter tolerances etc.

Algorithm type improvements

• Generic multi-stage algorithms should be made available through an algorithm union. The existence should be clearly documented.
• Making the previous point work may require relaxing type restrictions in the algorithm structs, which should be compensated by putting the restrictions in the (outer) constructors.

CTMRG environment initialization

Re-using environments seems okay at convergence phase, but not great at initialization.
We want to extend the options a bit here:

• Initialization from "product environment"
• Random initialization
• Initialization from previous iteration
• Initialization from previous iteration + gradient usage?

Associated to this, we want to improve stability and options for using non-FixedSpaceTruncation throughout the CTMRG scheme.

Normalization issues

During the optimization process, the norm of the PEPS tends to diverge, eventually leading to instabilities in the subalgorithms (and LAPACK errors).
There are several ways to mitigate this, the easiest consisting of breaking up the optimization into several smaller parts (checkpointing). At the breakpoints, we can renormalize, resymmetrize, etc without affecting the optimization procedure.
In the longer run, we should investigate if we can expose the inverse hessian in OptimKit such that we can achieve this directly in the finalize! of OptimKit.

Further ahead

• relax fixedpoint entry points?
• other fixedpoint equation schemes?
• algorithm explanations in docs?

(let me know if I missed something or messed something up).

[Yue-Zhengyuan]

Not the best place to ask, but I just want to know if you have any plan on adding GPU support to TensorKit (and PEPSKit)?

[lkdvos]

Definitely yes, we're trying to do it properly, and we have ideas on how to achieve it. It's just taking a bit of time since there are many other things going on as well.