Optimizing B

[wangzcl]

In original AA, one tries to minimize

$$\text{RSS}=||X-ABX||_F^2$$

In the optimization of $B$, $A$ is set fixed. Cutler and Breiman showed that this can be turned into a trivial convex least squares problem. In the current implementation (v0.6.2), it seems that the optimization of $B$ is not exactly the same as in C&B paper. It is the same as what is in Bauckhage and Thurau (2009) paper: minimizing $||A^+X-BX||_F^2$, i.e., finding archetypes that are closest to the "intermediate" or desired archetypes corresponding to current $A$. It seems that these two approaches are different and have different minimizers. The C&B approach always tries to minimize $\text{RSS}$, and ensures its convergence to local minimum. If we optimize a different objective function for $B$, does it still converge to a local minimum of $\text{RSS}$?

Did I miss anything?

[aleixalcacer]

Here https://www.jstatsoft.org/article/download/v030i08/312 it is described carefully how the algorithm is implemented.

However, if you have an alternative, we could discuss it :)

[wangzcl]

Thanks for the recommendation. It seems that both this paper and B&T 2009 paper used a same technique in optimizing $B$. I am still not sure if that approach is mathematically equivalent to the original design in C&B 1994. But it looks good as it works, and the original method also has its own problem and is harder to implement.

Let's keep it as it is. One thing we can do is to replace np.linalg.pinv(A) @ X with np.linalg.solve(A, X) which gives the same result but is usually faster.

[aleixalcacer]

If you provide some basic benchmarking showing it, +1 to replace the pinv :)