Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make grad / grad norm configurable #9

Open
spfrommer opened this issue Nov 14, 2023 · 1 comment
Open

Make grad / grad norm configurable #9

spfrommer opened this issue Nov 14, 2023 · 1 comment
Assignees
@spfrommer
Labels
enhancement New feature or request

Comments

@spfrommer
Copy link
Owner

Right now, all i/o gradients are logged using the grad norm, and parameter gradients are logged using the raw gradient. It would be nice to make this configurable.
@spfrommer spfrommer added the enhancement New feature or request label Nov 14, 2023
@spfrommer spfrommer self-assigned this Nov 14, 2023
@spfrommer
Copy link
Owner Author

Probably best implementation would be to have the log list take:
io_vals, io_grads, io_grad_norms, param_vals, param_grads, param_grad_norms

The attach parameter should be [vals, grads, grad_norms]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spfrommer spfrommer

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@spfrommer