Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using the Ask and tell interface does specifying num workers matter for the optimizer? #1028

Closed
smestern opened this issue Feb 2, 2021 · 2 comments

Comments

@smestern
Copy link

smestern commented Feb 2, 2021

Apologies this is less of an issue and more of a question.

When using the Ask and tell interface does specifying num workers matter for the optimizer?
In this case I am dealing with a large batch size of simulations that run in parallel (single neuron model fitting: up to 15000 points can be simulated) for 100 rounds. However the parallelization is handled by an external library (brian2), so I am simply using nevergrad in an ask tell format.

the basic code format is as follows

opt = ng.optimizers.TBPSA(params, num_workers=X, budget)
for x in rounds:
   points = []
   for x in batch_size:
         points.append(opt.ask())

   res = eval(points)

   for i, row in enumerate(res):
         opt.tell(row, points[i])

However, the documentation is not clear whether setting number of workers is crucial in this scenario. Does setting number of workers matter when not using the optimizer.optimize() function? Would it speed up convergence or is it simple used for parallelizing the function eval calls?
I imagine I have three options:

  1. Set number of workers to batch size (n=15000)
  2. Set number of workers to small safe number (n=10)
  3. Set number of workers to 1
@jrapin
Copy link
Contributor

jrapin commented Feb 3, 2021

num_workers is used by some of the algorithms, but not all of them:

  • TBPSA, DE and all population based optimizers: to set the initial size of the population, it must be at least the number of workers
  • NGOpt: to decide which sub-optimizer will be the most appropriate in those settings (NGOpt is sort of a "meta" optimizer)

In a nutshell it's better to provide a good enough order of magnitude for the parallelization but it's not absolutely crucial.
Does that make sense?

@smestern
Copy link
Author

smestern commented Feb 4, 2021

Absolutely that makes sense! Thank you!n

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants