Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TESTING] PR to test and work out new CUDA versions for CI #60

Merged

Conversation

mike-wendt
Copy link
Contributor

No description provided.

@tijyojwad
Copy link
Contributor

tijyojwad commented Aug 2, 2019

@mike-wendt questions about the new setup -

  1. The CUDA 10.1 py 3.6 jobs seems to failing because of a numpy installation error. Any chance the python in the conda env has numpy already?
    https://gpuci.gpuopenanalytics.com/blue/organizations/jenkins/gpuCI-private%2Fclara-genomics-analysis%2Fprb%2Fclara-genomics-analysis-cpu-build/detail/clara-genomics-analysis-cpu-build/869/pipeline

  2. The CUDA 9.0 GPU jobs seems to be using CUDA 10.1 based on the logs for cmake. Could it be that the symlink fr /usr/local/cuda in that image is pointing to /usr/local/cuda-10.1 instead of /usr/local/cuda-9.0?

-- Using CUDA 10.1 from /usr/local/cuda
-- Using CUDA 10.1 from /usr/local/cuda
  1. Are these new test targets only enabled for merges to master branch? Or all jobs for ClaraGenomicsAnalysis?

@raydouglass
Copy link
Member

  1. The CUDA 10.1 py 3.6 jobs seems to failing because of a numpy installation error. Any chance the python in the conda env has numpy already?
    https://gpuci.gpuopenanalytics.com/blue/organizations/jenkins/gpuCI-private%2Fclara-genomics-analysis%2Fprb%2Fclara-genomics-analysis-cpu-build/detail/clara-genomics-analysis-cpu-build/869/pipeline

Yes the conda environment has numpy already.

  1. The CUDA 9.0 GPU jobs seems to be using CUDA 10.1 based on the logs for cmake. Could it be that the symlink fr /usr/local/cuda in that image is pointing to /usr/local/cuda-10.1 instead of /usr/local/cuda-9.0?

Are you sure? Can you point to the logs?
I see this one and has the right CUDA version in cmake: https://gpuci.gpuopenanalytics.com/blue/organizations/jenkins/gpuCI-private%2Fclara-genomics-analysis%2Fprb%2Fclara-genomics-analysis-cpu-build/detail/clara-genomics-analysis-cpu-build/867/pipeline

  1. Are these new test targets only enabled for merges to master branch? Or all jobs for ClaraGenomicsAnalysis?

Triggering the CUDA 9.0 & 10.1 is manual right now. We want to iron out the builds before enabling it globally.

@tijyojwad
Copy link
Contributor

@raydouglass

Are you sure? Can you point to the logs?

https://gpuci.gpuopenanalytics.com/blue/rest/organizations/jenkins/pipelines/gpuCI-private/pipelines/clara-genomics-analysis/pipelines/prb/pipelines/clara-genomics-analysis-gpu-build/runs/431/log/?start=0

If you look at the CMake logs, it says using CUDA 10.1

Triggering the CUDA 9.0 & 10.1 is manual right now. We want to iron out the builds before enabling it globally.

Perfect, just wanted to make sure that out other merges won't be blocked till these issues are ironed out.

@tijyojwad tijyojwad changed the base branch from master to dev-v0.2.0 August 2, 2019 21:16
@tijyojwad tijyojwad force-pushed the tst-new-ci-cuda-vers branch 3 times, most recently from e22e710 to db80e17 Compare August 2, 2019 22:55
@tijyojwad
Copy link
Contributor

@raydouglass @mike-wendt made changes to support CUDA 9.0. can you re-trigger this PR with the different CUDA 9.0 and CUDA 10.1 jobs? I realized that what we'll actually need is CUDA 9.0, CUDA 9.2 and CUDA 10.1 ideally

- Disable logging module for CUDA files since there's a spdlog header
incompatibility with nvcc in CUDA < 9.2.
@tijyojwad tijyojwad force-pushed the tst-new-ci-cuda-vers branch from db80e17 to 43836f3 Compare August 2, 2019 23:03
@tijyojwad tijyojwad requested a review from ahehn-nv August 5, 2019 16:09
@tijyojwad tijyojwad requested a review from ahehn-nv August 5, 2019 16:54
@raydouglass
Copy link
Member

rerun tests

2 similar comments
@raydouglass
Copy link
Member

rerun tests

@raydouglass
Copy link
Member

rerun tests

@tijyojwad
Copy link
Contributor

New set of tests are passing. Failing tests are old status checks that can't be removed.

@tijyojwad tijyojwad merged commit 1e62243 into NVIDIA-Genomics-Research:dev-v0.2.0 Aug 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants