Extending Auto-sklearn can be incompatible with parallel execution #1290
Closed
Description
When running Auto-sklearn extended with a classifier/regressor etc with n_jobs>1
it can happen that the classifier/regressor Auto-sklearn was extended with is not available in the subprocess. Therefore, all function evaluations will crash.
This can be circumvented by calling add_classifier
etc in the subprocesses executing the machine learning runs. Sometimes, this is done automatically when for example the call to add_classifier
is done on the script's highest level and is not guarded by __name__ == "__main__"
. However, it would not work in a notebook for example.
Potential solution:
- Move the
_addons
attribute every component type has to a global addons registry. - Pass all registered addons to the workers
- When doing a function evaluations, check whether addons need to be registered on the worker and do so
Metadata
Assignees
Labels
No labels