Skip to content

Commit

Permalink
add DTensor to VLE and fix 2D sharding group in EBC (#2626)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: #2626

Adding DTensor state dict to VLE, this is identical to EBC DTensor path.

Future work of this diff is to consolidate the logic of EBC/PEA/VLE state dict into one parent class because of their significant similarities

A revert diff removed the 2D sharding logic in embedding bag collection mistakenly: D66800554, this diff adds it back in

Differential Revision: D65555595

fbshipit-source-id: ca12b8e833f9336e7c092b32d3955745806d9b56
  • Loading branch information
iamzainhuda authored and facebook-github-bot committed Dec 11, 2024
1 parent 33349ec commit e1b5edd
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion torchrec/distributed/embeddingbag.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
QuantizedCommCodecs,
ShardedTensor,
ShardingEnv,
ShardingEnv2D,
ShardingType,
ShardMetadata,
)
Expand Down Expand Up @@ -938,7 +939,11 @@ def _initialize_torch_state(self) -> None: # noqa
ShardedTensor._init_from_local_shards(
local_shards,
self._name_to_table_size[table_name],
process_group=self._env.process_group,
process_group=(
self._env.sharding_pg
if isinstance(self._env, ShardingEnv2D)
else self._env.process_group
),
)
)

Expand Down

0 comments on commit e1b5edd

Please sign in to comment.