Skip to content

Trying to make the code in RWKV more easily understood

Notifications You must be signed in to change notification settings

cooljoseph1/rwkv-simple

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RWKV Simple

The goal of this repository is to make the code for RWKV more easily understood. There's a few things I plan to do with this:

  1. Make the code readable. The code in the main repository is a big mess--it's got short, undescriptive names, hacks like "getattr" and "setattr", etc.
  2. Add documentation. Explain what the different parts of the model are.
  3. Write up a paper describing the algorithm in math terms.

Right now, all three of these still need to be done. Point (1) is the current priority.

About

Trying to make the code in RWKV more easily understood

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages