Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_download_url_archive_trailing_separator stalled #7615

Open
yarikoptic opened this issue Jun 14, 2024 · 1 comment
Open

test_download_url_archive_trailing_separator stalled #7615

yarikoptic opened this issue Jun 14, 2024 · 1 comment
Labels
spurious-test-failure Tests that fail unreliably test-failure For new or recurring failures of tests during CI runs

Comments

@yarikoptic
Copy link
Member

yarikoptic commented Jun 14, 2024 •
edited
Loading

while building debian package 0.19.6-3 in clean sid

  add (ok: 1)
  add-archive-content (ok: 1)
  download_url (ok: 1)
  save (ok: 1)
PASSED
datalad/local/tests/test_download_url.py::test_download_url_archive_trailing_separator create(ok): . (dataset)

never returned. ps auxw -H seems to show

neurode+ 1248228  0.0  0.0   2860  2048 pts/31   S+   10:59   0:00                                 /usr/bin/make -f debian/rules override_dh_auto_test
neurode+ 1248229  0.0  0.0   2600  1536 pts/31   S+   10:59   0:00                                   /bin/sh -c PATH=/build/datalad-0.19.6/bin:$PATH dh_auto_test -- --test-pytest
neurode+ 1248230  0.0  0.0  20404 18048 pts/31   S+   10:59   0:00                                     /usr/bin/perl /usr/bin/dh_auto_test -- --test-pytest
neurode+ 1248244  0.0  0.0  21096 17008 pts/31   S+   10:59   0:00                                       /usr/bin/python3 /usr/bin/pybuild --test --test-pytest -i python{version} -p 3.12 3.11 --test-pytest
neurode+ 1248650  0.0  0.0   2600  1536 pts/31   S+   10:59   0:00                                         /bin/sh -c cd /build/datalad-0.19.6/.pybuild/cpython3_3.12_datalad/build; python3.12 -m pytest -s -v -k 'not test_test and not test_subprocess_return_code_capture and not test_gracefull_death' -c /build/datalad-0.19.6/tox.ini
neurode+ 1248651  3.7  0.0 1246296 129088 pts/31 Sl+  10:59   2:11                                           python3.12 -m pytest -s -v -k not test_test and not test_subprocess_return_code_capture and not test_gracefull_death -c /build/datalad-0.19.6/tox.ini
neurode+ 1249428  0.0  0.0 304716 71952 pts/31   Sl+  10:59   0:00                                             python3.12 -m pytest -s -v -k not test_test and not test_subprocess_return_code_capture and not test_gracefull_death -c /build/datalad-0.19.6/tox.ini
neurode+ 1959829  0.0  0.0   7228  3584 pts/31   S+   11:15   0:00                                             git -c diff.ignoreSubmodules=none -c core.quotepath=false annex addurl --relaxed --with-files --json --json-error-messages --batch
neurode+ 1959834  0.0  0.0 1074069700 57856 pts/31 Sl+ 11:15   0:00                                               /usr/bin/git-annex addurl --relaxed --with-files --json --json-error-messages --batch
neurode+ 1959847  0.0  0.0   7228  3840 pts/31   S+   11:15   0:00                                                 git --git-dir=.git --work-tree=. --literal-pathspecs cat-file --batch
neurode+ 1962509  0.0  0.0 1254536 115604 pts/31 Sl+  11:15   0:00                                             python3.12 -m pytest -s -v -k not test_test and not test_subprocess_return_code_capture and not test_gracefull_death -c /build/datalad-0.19.6/tox.ini
neurode+ 1963290  0.0  0.0      0     0 pts/31   Z+   11:15   0:00                                             [git] <defunct>

so might relate to that defunct git may be?

I think we should start using https://pypi.org/project/pytest-timeout/ to prevent such stalls and error out late but still do.

For now I will just try to rerun.

edit: on rerun stalled again. Skipping for now!
@yarikoptic yarikoptic added spurious-test-failure Tests that fail unreliably test-failure For new or recurring failures of tests during CI runs labels Jun 14, 2024
@yarikoptic
Copy link
Member Author

might have been just a symptom of something more grave since now first FAILs and then stalls with

datalad/support/tests/test_annexrepo.py::test_AnnexRepo_get_contentlocation[True] PASSED
datalad/support/tests/test_annexrepo.py::test_AnnexRepo_addurl_to_file_batched FAILED
datalad/support/tests/test_annexrepo.py::test_annexrepo_fake_dates_disables_batched

neurode+ 3064295  0.0  0.0 304684 69664 pts/29   Sl+  12:55   0:00                                             python3.12 -m pytest -s -v -k not test_test and not test_subprocess_return_code_capture and not test_gracefull_death and not test_download_url_archive_trailing_separator -c /build/datalad-0.19.6/tox.ini
neurode+ 3528841  0.0  0.0      0     0 pts/29   Z+   13:15   0:00                                             [python3.12] <defunct>
neurode+ 3537706  0.0  0.0   7228  3328 pts/29   S+   13:15   0:00                                             git -c diff.ignoreSubmodules=none -c core.quotepath=false annex addurl --with-files --json --json-error-messages --batch
neurode+ 3537711  0.0  0.0 1074069700 57728 pts/29 Sl+ 13:15   0:00                                               /usr/bin/git-annex addurl --with-files --json --json-error-messages --batch
neurode+ 3537720  0.0  0.0   7228  4096 pts/29   S+   13:15   0:00                                                 git --git-dir=.git --work-tree=. --literal-pathspecs cat-file --batch

interestingly the build which was pure rebuild of -2 without "cutting of" the network through bad proxy:

diff -Nru datalad-0.19.6/debian/rules datalad-0.19.6/debian/rules
--- datalad-0.19.6/debian/rules 2024-03-10 19:12:04.000000000 -0400
+++ datalad-0.19.6/debian/rules 2024-06-14 12:48:01.000000000 -0400
@@ -7,10 +7,9 @@
 # test_test would be removed as 0.17.1
 # test_subprocess_return_code_capture - https://github.com/datalad/datalad/issues/6109
 # test_gracefull_death - too flaky https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1029450
-export PYBUILD_TEST_ARGS = -s -v -k 'not test_test and not test_subprocess_return_code_capture and not test_gracefull_death' -c $(CURDIR)/tox.ini
+# test_download_url_archive_trailing_separator - stalls https://github.com/datalad/datalad/issues/7615
+export PYBUILD_TEST_ARGS = -s -v -k 'not test_test and not test_subprocess_return_code_capture and not test_gracefull_death and not test_download_url_archive_trailing_separator' -c $(CURDIR)/tox.ini
 export DATALAD_TESTS_NONETWORK = 1
-export http_proxy=
-export https_proxy=

did not stall! checking this build which stalls we have a number of failing tests

neurodebian@smaug ~/deb/builds/datalad/0.19.6-3 % grep test.*FAIL *build
datalad/distribution/tests/test_siblings.py::test_siblings FAILED
datalad/local/tests/test_addurls.py::TestAddurls::test_addurls_create_newdataset FAILED
datalad/support/tests/test_annexrepo.py::test_AnnexRepo_web_remote FAILED
datalad/support/tests/test_annexrepo.py::test_AnnexRepo_backend_option FAILED
datalad/support/tests/test_annexrepo.py::test_AnnexRepo_addurl_to_file_batched FAILED

rabbit hole...

@yarikoptic yarikoptic mentioned this issue Jun 14, 2024
Draft
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
spurious-test-failure Tests that fail unreliably test-failure For new or recurring failures of tests during CI runs
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@yarikoptic