Does jax.experimental.shard_map.shard_map(..., auto=...)
implement xla auto-sharding?
#24736
Unanswered
man2machine
asked this question in
Q&A
Replies: 1 comment
-
I have the same question. Maybe this pull request is enabling Alpa. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Does the
auto
argument call the experimental XLA auto-sharding algorithm in https://github.com/openxla/xla/tree/main/xla/hlo/experimental/auto_sharding which is based on the Alpa paper https://github.com/alpa-projects/alpaIf so, when auto-sharding is performed, does it perform only shard-parallelism (intra-operator) or pipeline-parallelism (inter-operator) as well as described in the Alpa paper?
Beta Was this translation helpful? Give feedback.
All reactions