You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I understand that when using Horovod in a multi node environment, I should have each shard dedicated per node. Based on the size of my compute node, I've split the data into 4 shards - based on the max that can fit on my node.
I'm wondering if it's possible to have more shards than the # of nodes? For example, if I have 2 nodes, can each download & process a dataset that has 4 shards? In my use case, each shard is big enough to fit on a single worker, but I do not want to have 4 nodes. Can I make do with 2 workers that process the shards in something like a round robin approach?
Thanks
The text was updated successfully, but these errors were encountered:
Hi!
I understand that when using Horovod in a multi node environment, I should have each shard dedicated per node. Based on the size of my compute node, I've split the data into 4 shards - based on the max that can fit on my node.
I'm wondering if it's possible to have more shards than the # of nodes? For example, if I have 2 nodes, can each download & process a dataset that has 4 shards? In my use case, each shard is big enough to fit on a single worker, but I do not want to have 4 nodes. Can I make do with 2 workers that process the shards in something like a round robin approach?
Thanks
The text was updated successfully, but these errors were encountered: