Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SETUP.md to replace the correct random string #182

Merged
merged 6 commits into from
Feb 28, 2019
Merged

Update SETUP.md to replace the correct random string #182

merged 6 commits into from
Feb 28, 2019

Conversation

slochower
Copy link
Collaborator

The sed command in SETUP.md was not using the appropriate random string in deploy.sh.

@slochower
Copy link
Collaborator Author

slochower commented Feb 27, 2019

Something weird is happening with psutil on Travis.

  running build_ext
  building 'psutil._psutil_linux' extension
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/psutil
  x86_64-conda_cos6-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/python_1551150806669/work=/usr/local/src/conda/python-3.6.7 -fdebug-prefix-map=/home/travis/miniconda/envs/manubot=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/python_1551150806669/work=/usr/local/src/conda/python-3.6.7 -fdebug-prefix-map=/home/travis/miniconda/envs/manubot=/usr/local/src/conda-prefix -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=551 -DPSUTIL_LINUX=1 -I/home/travis/miniconda/envs/manubot/include/python3.6m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.6/psutil/_psutil_common.o
  unable to execute 'x86_64-conda_cos6-linux-gnu-gcc': No such file or directory
  error: command 'x86_64-conda_cos6-linux-gnu-gcc' failed with exit status 1

@slochower
Copy link
Collaborator Author

Possible we may need conda install gxx_linux-64to compile psutil.
RcppCore/Rcpp#770

@slochower
Copy link
Collaborator Author

Also possible that conda update conda is doing something with libgcc that is breaking something:
travis-ci/travis-ci#8512

@slochower
Copy link
Collaborator Author

@dhimmel Have you seen this Travis error before?

@dhimmel
Copy link
Member

dhimmel commented Feb 27, 2019

Have you seen this Travis error before?

No.

Also possible that conda update conda is doing something with libgcc that is breaking something:

We don't run conda update conda in:

rootstock/.travis.yml

Lines 7 to 17 in 2a4c813

before_install:
- wget https://repo.continuum.io/miniconda/Miniconda3-4.5.12-Linux-x86_64.sh
--output-document miniconda.sh
- bash miniconda.sh -b -p $HOME/miniconda
- source $HOME/miniconda/etc/profile.d/conda.sh
- hash -r
- conda config --set always_yes yes --set changeps1 no
install:
- conda env create --quiet --file build/environment.yml
- conda list --name manubot
- conda activate manubot

I'll have more time to look tomorrow, but I've restarted the build in case it's temporary. Otherwise, perhaps we can add a workaround, as suggested in the other issues, to this PR.

@slochower
Copy link
Collaborator Author

Thanks. Also failing on a clone of this repository I made last night for a new manuscript, so it's not unique.

@dhimmel
Copy link
Member

dhimmel commented Feb 28, 2019

Thanks. Also failing on a clone of this repository I made last night for a new manuscript, so it's not unique.

This is bad. Perhaps we want to enable daily cron builds of rootstock so we can more quickly discover and pinpoint these issues. This is disappointing because we have tried to pin as much as possible regarding our conda environment. However, the environment is still changing (and breaking). A good reason for us to switch to docker where CI runs don't have to rebuild the env each time.

@dhimmel
Copy link
Member

dhimmel commented Feb 28, 2019

Possible we may need conda install gxx_linux-64to compile psutil.

Turned out installing conda's precompiled psutil (rather than pip) fixed the issue.

@dhimmel dhimmel merged commit 717630e into manubot:master Feb 28, 2019
dhimmel pushed a commit that referenced this pull request Feb 28, 2019
This build is based on
717630e.

This commit was created by the following Travis CI build and job:
https://travis-ci.com/manubot/rootstock/builds/102622378
https://travis-ci.com/manubot/rootstock/jobs/181210990

[ci skip]

The full commit message that triggered this build is copied below:

Fix SETUP.md random string & psutil install

Merges #182

Update SETUP.md to replace the correct random string

Fix missing x86_64-conda_cos6-linux-gnu-gcc in psutil install by installing
psutil using conda.

.travis.yml: one configuration option per line (non-functional whitespace change)
dhimmel pushed a commit that referenced this pull request Feb 28, 2019
This build is based on
717630e.

This commit was created by the following Travis CI build and job:
https://travis-ci.com/manubot/rootstock/builds/102622378
https://travis-ci.com/manubot/rootstock/jobs/181210990

[ci skip]

The full commit message that triggered this build is copied below:

Fix SETUP.md random string & psutil install

Merges #182

Update SETUP.md to replace the correct random string

Fix missing x86_64-conda_cos6-linux-gnu-gcc in psutil install by installing
psutil using conda.

.travis.yml: one configuration option per line (non-functional whitespace change)
@agitter
Copy link
Member

agitter commented Feb 28, 2019

It would be nice if there was a way to use conda to help us locate which versions of the unpinned packaged had changed between builds. In this case, psutil was updated from 5.5.0 (last successful build) to 5.5.1 (failed build) on PyPI on February 15. We don't want to create that infrastructure though.

Docker may indeed be the best solution here. I expect we would still want to maintain a functional conda environment but could use Docker for reliable builds.

@agitter
Copy link
Member

agitter commented Feb 28, 2019

I'm now realizing that it may not be as simple as the version of a single package. Other manubot builds are still passing even with psutil 5.5.1.

@slochower
Copy link
Collaborator Author

Docker may indeed be the best solution here. I expect we would still want to maintain a functional conda environment but could use Docker for reliable builds.

I've used Docker a few times in the past, but not within the past year or so. Is there a simple recommended way for package maintainers to effectively say to users: "Here is our Docker image, to use it execute this one line"? Or is it still a multistep process with potential permissions issues (still need sudo access on linux/macOS)? Have either of you used Singularity? I think our large clusters here have largely migrated from Docker to Singularity.

@agitter
Copy link
Member

agitter commented Mar 20, 2019

Is there a simple recommended way for package maintainers to effectively say to users: "Here is our Docker image, to use it execute this one line"?

@slochower I am no Docker expert, but in my limited experience I'd say that yes it easy to do that if you already have Docker configured on your system.

Have either of you used Singularity?

I haven't used Singularity much. I too have seen a lot of clusters migrating to Singularity. My understanding is that Singularity can run Docker containers so there is no major downside to creating a Docker container for Singularity or Docker users.

adebali pushed a commit to CompGenomeLab/lemur-manuscript-archive that referenced this pull request Mar 4, 2020
Merges manubot/rootstock#182

Update SETUP.md to replace the correct random string

Fix missing x86_64-conda_cos6-linux-gnu-gcc in psutil install by installing
psutil using conda.

.travis.yml: one configuration option per line (non-functional whitespace change)
ploegieku added a commit to ploegieku/2023-functional-homology-paper that referenced this pull request Aug 6, 2024
Merges manubot/rootstock#182

Update SETUP.md to replace the correct random string

Fix missing x86_64-conda_cos6-linux-gnu-gcc in psutil install by installing
psutil using conda.

.travis.yml: one configuration option per line (non-functional whitespace change)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants