Skip to content

Commit

Permalink
Document difference in failing and skipping
Browse files Browse the repository at this point in the history
  • Loading branch information
robertodr committed Sep 21, 2018
1 parent e0b6273 commit 56ce4f3
Show file tree
Hide file tree
Showing 9 changed files with 57 additions and 32 deletions.
32 changes: 12 additions & 20 deletions .appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,34 +8,26 @@ build:
cache:
- c:\tools\vcpkg\installed\ -> testing\dependencies\appveyor\install.bat
- c:\msys64\var\cache\pacman\pkg -> testing\dependencies\appveyor\install.bat
- c:\Deps\boost_1_67_0 -> testing\dependencies\appveyor\boost.bat
- c:\Deps\conda\ -> testing\dependencies\appveyor\anaconda.ps1
- c:\Deps\boost_1_67_0 -> testing\dependencies\appveyor\boost.bat

image:
- Visual Studio 2017

environment:
# Create expected SHELL variable for pipenv.
SHELL: "windows"
CTEST_OUTPUT_ON_FAILURE: "1"
SHELL: 'windows'
CTEST_OUTPUT_ON_FAILURE: '1'
matrix:
- CMAKE_GENERATOR: "MSYS Makefiles"
BUILDFLAGS: "VERBOSE=1"
CMAKEARGS: ""
- CMAKE_GENERATOR: "Visual Studio 15 2017 Win64"
BUILDFLAGS: "/verbosity:normal"
CMAKEARGS: ""
- CMAKE_GENERATOR: "Ninja"
BUILDFLAGS: "-v"
CMAKEARGS: ""
- CMAKE_GENERATOR: "MSYS Makefiles"
ANACONDA_TESTS_ONLY: "1"
BUILDFLAGS: "VERBOSE=1"
CMAKEARGS: ""
- CMAKE_GENERATOR: "Visual Studio 15 2017 Win64"
ANACONDA_TESTS_ONLY: "1"
BUILDFLAGS: "/verbosity:normal"
CMAKEARGS: ""
- CMAKE_GENERATOR: 'MSYS Makefiles'
BUILDFLAGS: 'VERBOSE=1'
- CMAKE_GENERATOR: 'Visual Studio 15 2017 Win64'
BUILDFLAGS: '/verbosity:normal'
- CMAKE_GENERATOR: 'Ninja'
BUILDFLAGS: '-v'
- CMAKE_GENERATOR: 'Visual Studio 15 2017 Win64'
BUILDFLAGS: '/verbosity:normal'
ANACONDA_TESTS_ONLY: '1'

matrix:
fast_finish: true
Expand Down
12 changes: 8 additions & 4 deletions chapter-11/recipe-04/cxx-example/custom.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,23 @@ cp ../example.cpp .
if [[ "$OSTYPE" == "msys" ]]; then
conda.exe config --set always_yes yes --set changeps1 no

conda.exe build conda-recipe
conda.exe build --no-anaconda-upload conda-recipe

conda.exe install --use-local conda-example-simple
conda.exe install --no-update-dependencies --use-local --yes conda-example-simple

hello-conda.exe

conda.exe clean --all --yes
else
PATH=$HOME/Deps/conda/bin${PATH:+:$PATH}

conda build conda-recipe
conda build --no-anaconda-upload conda-recipe

conda install --use-local conda-example-simple
conda install --no-update-deps --use-local --yes conda-example-simple

hello-conda

conda clean --all --yes
fi

exit $?
2 changes: 1 addition & 1 deletion chapter-11/recipe-04/cxx-example/menu.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
local:
env:
- VERBOSE_OUTPUT: 'True'
- VERBOSE_OUTPUT: 'ON'
1 change: 1 addition & 0 deletions chapter-11/recipe-05/cxx-example/conda-recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ requirements:
- cmake >=3.5
- {{ compiler('cxx') }}
host:
- intel-openmp 2018
- mkl-devel 2018

about:
Expand Down
12 changes: 8 additions & 4 deletions chapter-11/recipe-05/cxx-example/custom.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,23 @@ cp ../example.cpp .
if [[ "$OSTYPE" == "msys" ]]; then
conda.exe config --set always_yes yes --set changeps1 no

conda.exe build conda-recipe
conda.exe build --no-anaconda-upload conda-recipe

conda.exe install --use-local conda-example-dgemm
conda.exe install --no-update-deps --use-local --yes conda-example-dgemm

dgemm-example.exe

conda.exe clean --all --yes
else
PATH=$HOME/Deps/conda/bin${PATH:+:$PATH}

conda build conda-recipe
conda build --no-anaconda-upload conda-recipe

conda install --use-local conda-example-dgemm
conda install --no-update-deps --use-local --yes conda-example-dgemm

dgemm-example

conda clean --all --yes
fi

exit $?
6 changes: 5 additions & 1 deletion chapter-11/recipe-05/cxx-example/menu.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
appveyor-vs:
env:
- VERBOSE_OUTPUT: 'ON'

local:
env:
- VERBOSE_OUTPUT: 'True'
- VERBOSE_OUTPUT: 'ON'
5 changes: 3 additions & 2 deletions contributing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ To update the README files, run `python contributing/generate-readmes.py` - this

#### Indentation

We use 2 spaces instead of 4 spaces to reduce the printed page width.
No tabs.
We use 2 spaces instead of 4 spaces to reduce the printed page width. No tabs.
Get the integration for [EditorConfig](https://editorconfig.org/) in your
favorite editor to help you keep the conventions.


#### Case of commands
Expand Down
16 changes: 16 additions & 0 deletions testing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,22 @@ appveyor-msys:
- 'MSYS Makefiles'
```
## When to use `failing_generators` and `skip_generators`

Marking generators as expected failures or skipping them achieves more or less
the same goal: not failing CI for recipes that are known not to work under
certain conditions. Using `failing_generators` means that the recipe is actually
tested, but the possible failure is not elevated to an error; whereas using
`skip_generators` will skip the testing altogether.
The semantic to differentiate the use of the two is thus:
1. Use `failing_generators` if the recipe does not work under the current CI set
up, but _it could be made_ to work. As an example, the Ninja is always marked as
an expected failure for Fortran recipes, since its set up is rather contrived,
requiring specific versions specific versions of CMake _and_ Ninja.
The hurdle could however be overcome in the future.
2. Use `skip_generators` when _you cannot foresee any way_ to make the recipe work.
This is the case for the Fortran recipes with the Visual Studio generators.


## Bulding targets

Expand Down
3 changes: 3 additions & 0 deletions testing/collect_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ def run_command(*, step, command, expect_failure):
# Stream stderr always
stderr_streamer = functools.partial(streamer, file_handle=sys.stderr)
stderr = ''
# subprocess.Popen can be managed as a context and allows us to stream
# stdout and stderr in real-time
with subprocess.Popen(
cmd,
bufsize=1,
Expand Down Expand Up @@ -166,6 +168,7 @@ def run_example(topdir, generator, ci_environment, buildflags, recipe, example):
custom_script = 'custom.sh'
custom_script_path = cmakelists_path / custom_script
if custom_script_path.exists():
sys.stdout.write('\nRunning a custom.sh script\n')
# if this directory contains a custom.sh script, we launch it
step = custom_script
command = 'bash "{0}" "{1}"'.format(custom_script_path, build_directory)
Expand Down

0 comments on commit 56ce4f3

Please sign in to comment.