Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does Tapir support vector mode? #187

Closed
gdalle opened this issue Jun 20, 2024 · 3 comments
Closed

Does Tapir support vector mode? #187

gdalle opened this issue Jun 20, 2024 · 3 comments

Comments

@gdalle
Copy link

gdalle commented Jun 20, 2024

By "vector mode" I mean the ability to propagate a chunk of seeds in the pullback, similar to ForwardDiff's Chunk or Enzyme's BatchDuplicated.
Thanks to JuliaDiff/DifferentiationInterface.jl#320, DifferentiationInterface will use vector mode to speed up (dense & sparse) Jacobians and Hessians.

@yebai
Copy link
Contributor

yebai commented Jun 20, 2024

I will leave this to @willtebbutt for an answer.

I have a question about vector mode in general. Given that Tapir supports input—and data-dependent control flows, what happens if the forward pass takes different paths for different inputs?

@gdalle
Copy link
Author

gdalle commented Jun 20, 2024

In autodiff parlance, vector mode refers to propagating several pullback seeds dy1, ..., dyk with the same input x. In other words, the control flow (which is dictated by x) remains unchanged

@willtebbutt
Copy link
Member

Currently no support for vector mode I'm afraid. You'll have to fall back to just running the forwards- and reverse-passes several times.

@yebai yebai closed this as not planned Won't fix, can't repro, duplicate, stale Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants