Skip to content

Commit

Permalink
No public description
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 568424151
  • Loading branch information
DeepMind LMI Team authored and david-lindner committed Nov 13, 2023
1 parent 69c993c commit 71c33e2
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions tracr/craft/transformers.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ def __post_init__(self):
assert self.w_ov.output_space.issubspace(self.residual_space)

def apply(self, x: bases.VectorInBasis) -> bases.VectorInBasis:
assert self.residual_space is not None
assert x in self.residual_space
# seq_len x query_space
queries = x.project(self.w_qk.left_space)
Expand Down

0 comments on commit 71c33e2

Please sign in to comment.