torch.optim.Optimizer vs. torchopt.Optimizer

[StefanoWoerner]

What are the differences and advantages of torchopt.Optimizer and its subclasses compared with torch.optim.Optimizer? It is somewhat unclear to me why it exists and why I should or should not use it. Thanks!

[XuehaiPan]

The main difference between torchopt.Optimizer and torch.optim.Optimzier is their composability. You can differentiate through the torchopt.Optimizer with inplace=False while updating the parameters. This feature is partially supported in PyTorch's torch.optim.Optimizer for higher-order algorithms.

[StefanoWoerner]

Okay, got it.But then what is the difference between torchopt.Optimizer and torchopt.MetaOptimizer? I thought the the difference was that torchopt.MetaOptimizer is differentiable.