Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Install PyTorch from conda-forge channel rather than pytorch #30497

Merged
merged 4 commits into from
Dec 18, 2024

Conversation

lesteve
Copy link
Member

@lesteve lesteve commented Dec 17, 2024

Fix #30390.

Copy link

github-actions bot commented Dec 17, 2024

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: 04557fe. Link to the linter CI: here

@lesteve
Copy link
Member Author

lesteve commented Dec 17, 2024

@scikit-learn-bot update lock-files --select-tag cuda

@betatim
Copy link
Member

betatim commented Dec 17, 2024

Something is weird no? Because the .lock file still contains pytorch from the pytorch channel :-/

@lesteve
Copy link
Member Author

lesteve commented Dec 17, 2024

Yeah I was hoping the @scikit-learn-bot magic comment would work but it looks like it does not, let me try to update locally and push.

Some files are downloaded directly from main and that includes update_environment_and_lock_files.py which you know make the @scikit-learn-bot not that useful ... see build log

This was done to prevent abuse, see #29505 (comment). This would need a closer look, in my opinion this makes the @scikit-learn-bot not super useful, you can only update the lock-files without changing anything but maybe it's safer to be conservative. The only use case I can see with the current behaviour is that you need the latest version of a package that is already in the lock-file and you can not wait until next Monday when lock-files get automatically updated. It can not be used in the vast majority of case, for example when you need a new package, or need to tweak an existing environment or lock-file.

@lesteve
Copy link
Member Author

lesteve commented Dec 17, 2024

By the way @betatim do you have automated alarms set-up for detecting CUDA activity in scikit-learn? Or you bumped into the PR by chance?

@lesteve
Copy link
Member Author

lesteve commented Dec 17, 2024

Looks like pytorch-gpu is installed and the test pass see build log

I am going to retrigger it with pytest -v just to check the gpu tests did run. Looks like they did run see build log you can see some test names with cuda with PASSED (and not SKIPPED):

conda/envs/sklearn/lib/python3.12/site-packages/sklearn/decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='full',whiten=True)-check_array_api_input_and_values-torch-cuda-float64] PASSED [  1%]

@betatim
Copy link
Member

betatim commented Dec 17, 2024

By the way @betatim do you have automated alarms set-up for detecting CUDA activity in scikit-learn? Or you bumped into the PR by chance?

The 👁️ of CUDA makes my watch beep when there is a GPU issue or PR ;)

Just happened to be the top notification when I started looking at things

Copy link
Member

@betatim betatim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you find the -v useful in the pytest command let's keep it. I think the CI output should be so that if people look at it to debug/understand what happened it should by default give the infos they need (without totally overwhelming readers with a million lines of output). And it looks like you wanted to -v to see that things worked as expected.

@lesteve
Copy link
Member Author

lesteve commented Dec 17, 2024

I am fine with keeping -v, since we are running only the array API tests the log isn't horribly longer.

More about the context: I wanted to at least try with -v to be reassured that somehow the CUDA tests were indeed running. I was slightly afraid that somehow the CUDA tests would be skipped without me noticing. Maybe part of it is that I am not very familiar with how array API tests are structured.

@thomasjpfan thomasjpfan merged commit ebfda90 into scikit-learn:main Dec 18, 2024
34 checks passed
@lesteve lesteve deleted the pytorch-conda-forge branch January 8, 2025 04:18
@lesteve lesteve mentioned this pull request Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI Replace pytorch conda channel in CI lock-files
4 participants