What's the different with Liger-Kernel

[CharlesJhonson]

Please describe your question

I noticed that there is an open source project called Liger-Kernel:[https://github.com/linkedin/Liger-Kernel], which is also a triton operator collection library. What is the difference between it and gems?

[iclementine]

We implement a broader range of kernels, many of which are "standard" kernels for general purpose.

1. elementwise, reduce, scan for general n-dimensional tensor.
2. some kernels that are also for general purpose but are a little challenging in triton, like sort, topk, unique, etc, mainly search&sort related.

We also have kernels targeting neural networks, like RoPE, which are common in LLM related libraries.

[iclementine]

Also, we are working on supporting more accelerators in a single source library.

[CharlesJhonson]

Ok, thank you very much!
In addition, I noticed that many operators only have forward propagation implementation functions, such as RMS_norm. If it is used for LLM training, how should its backward propagation be implemented? @iclementine