[#430] Allow-failures for old GHC on Stack at TravisCI

Resolves #430
kowainik · chshersh · Mar 22, 2020 · Mar 22, 2020 · Mar 22, 2020 · e871661c2f9d4cfe2e54781661bda0ca685454f1
commit e871661c2f9d4cfe2e54781661bda0ca685454f1
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -20,6 +20,7 @@ The changelog is available [on GitHub][2].
   [#429](https://github.com/kowainik/summoner/issues/429):
   Support GHC-8.8.3 in the project.
   Make GHC-8.8.3 default for the generated projects.
+  Upgrade Stack LTS to `15.5`.
   (by [@vrom911](https://github.com/vrom911))
 * Upgrade GHC-8.6.5 LTS to `14.27`.
   (by [@vrom911](https://github.com/vrom911))
@@ -100,6 +101,11 @@ The changelog is available [on GitHub][2].
   __#CLI__ __#TUI__
   Implement non-interactive mode with `-n | --non-interacive` command-line option.
   (by [@vrom911](https://github.com/vrom911))
+* [#430](https://github.com/kowainik/summoner/issues/430):
+  Instead of not including old GHCs for Stack check at Travis,
+  those versions are added to the `allow_failures` section of matrix.
+
+  _Note:_ For the Stack failure reasons see [this issue](https://github.com/commercialhaskell/stack/issues/4488).
 
 ## 1.4.0.0 – Dec 25, 2019 🎅
 

diff --git a/README.md b/README.md
@@ -99,11 +99,11 @@ Features related to the structure and content of the generated projects.
 + Option to include an alternative prelude, if desired. The project would then
   use the [`mixins` technique](https://www.reddit.com/r/haskelltil/comments/9qa366/easy_way_to_replace_default_prelude_with_the/).
 + Whole Hackage-upload checklist support (exhaustive `.cabal` file, PVP versioning, GHC options, conventional meta files).
-+ Support for multiple GHC versions, with thoughtful reflection on project meta, base versions (e.g. `base >= 4.9 && < 4.12`), etc.
++ Support for multiple GHC versions, with thoughtful reflection on project meta, base versions (e.g. `base >= 4.9 && < 4.13`), etc.
 + Ability to create runnable Haskell scripts.
-+ Different license support: MIT, BSD2, BSD3, GPL-2, GPL-3, LGPL-2.1, LGPL-3, AGPL-3, Apache-2.0, MPL-2.0, None (All Rights Reserved license without file).
++ Different license support: MIT, BSD-2-Clause, BSD-3-Clause, GPL-2.0, GPL-3.0, LGPL-2.1, LGPL-3.0, AGPL-3.0, Apache-2.0, MPL-2.0, ISC, None (without file).
 + Creation of the `CHANGELOG.md` file with [PVP versioning policy](https://pvp.haskell.org).
-+ Ability to include any custom files (including `.stylish-haskell.yaml`, `CONTRIBUTING.md`, `CODEOWNERS`, etc.).
++ Ability to include any custom files (including `.stylish-haskell.yaml`, `CONTRIBUTING.md`, `CODEOWNERS`, `FUNDING.yml` etc.).
 + Usage of the `ghc-options` field with sensible defaults.
 
   If `ghc-options` are not explicitly stated in the configuration file, then the following list of GHC flags is added to all stanzas:
@@ -600,7 +600,7 @@ main = putStrLn "Hello, World!"
 ```haskell
 #!/usr/bin/env stack
 {- stack
-  --resolver lts-14.18
+  --resolver lts-15.5
   script
   --package base
 -}
@@ -638,6 +638,21 @@ We can not change or configure this behaviour, but there are several workarounds
    export HUB_PROTOCOL="https"
    ```
 
+<hr>
+
+> Why some of the Stack checks on my TravisCI matrix are marked as "This job is
+> allowed to fail"?
+
+Due to the Stack problem of working with older versions of Cabal, the build can
+fail with Stack for some older GHCs. We are adding these checks into
+`allow_failures` section, as this is a known issue, and there is no need for the
+whole build to fail because of that.
+
+For more information about the issue and some workarounds from Stack developers,
+see the following ticket:
+
+  * [commersialhaskell/stack issue](https://github.com/commercialhaskell/stack/issues/4488)
+
 ## For Developers [↑](#structure)
 
 If you'd like to take part in the development processes, here are a few things to keep in mind:

diff --git a/stack.yaml b/stack.yaml
@@ -1,10 +1,9 @@
-resolver: lts-15.4
+resolver: lts-15.5
 
 packages:
   - summoner-cli/
   - summoner-tui/
 
 extra-deps:
-  - colourista-0.0.0.0
   - generic-data-0.8.0.0
   - hspec-hedgehog-0.0.1.1
diff --git a/summoner-cli/examples/cabal-full/.travis.yml b/summoner-cli/examples/cabal-full/.travis.yml
@@ -17,6 +17,7 @@ matrix:
   - ghc: 8.6.5
   - ghc: 8.8.3
 
+
 
 install:
   # HLint check

diff --git a/summoner-cli/examples/full-batteries/.travis.yml b/summoner-cli/examples/full-batteries/.travis.yml
@@ -20,6 +20,9 @@ matrix:
   - ghc: 8.6.5
   - ghc: 8.8.3
 
+  - ghc: 8.0.2
+    env: STACK_YAML="$TRAVIS_BUILD_DIR/stack-8.0.2.yaml"
+
   - ghc: 8.2.2
     env: STACK_YAML="$TRAVIS_BUILD_DIR/stack-8.2.2.yaml"
 
@@ -31,6 +34,17 @@ matrix:
 
   - ghc: 8.8.3
     env: STACK_YAML="$TRAVIS_BUILD_DIR/stack.yaml"
+
+  allow_failures:
+
+  - ghc: 8.0.2
+    env: STACK_YAML="$TRAVIS_BUILD_DIR/stack-8.0.2.yaml"
+
+  - ghc: 8.2.2
+    env: STACK_YAML="$TRAVIS_BUILD_DIR/stack-8.2.2.yaml"
+
+  - ghc: 8.4.4
+    env: STACK_YAML="$TRAVIS_BUILD_DIR/stack-8.4.4.yaml"
 
 install:
   # HLint check

diff --git a/summoner-cli/examples/full-batteries/stack.yaml b/summoner-cli/examples/full-batteries/stack.yaml
@@ -1 +1 @@
-resolver: lts-15.4
+resolver: lts-15.5
diff --git a/summoner-cli/examples/stack-full/.travis.yml b/summoner-cli/examples/stack-full/.travis.yml
@@ -24,6 +24,11 @@ matrix:
 
   - ghc: 8.8.3
     env: STACK_YAML="$TRAVIS_BUILD_DIR/stack.yaml"
+
+  allow_failures:
+
+  - ghc: 8.4.4
+    env: STACK_YAML="$TRAVIS_BUILD_DIR/stack-8.4.4.yaml"
 
 install:
   # HLint check

diff --git a/summoner-cli/examples/stack-full/stack.yaml b/summoner-cli/examples/stack-full/stack.yaml
@@ -1 +1 @@
-resolver: lts-15.4
+resolver: lts-15.5
diff --git a/summoner-cli/src/Summoner/GhcVer.hs b/summoner-cli/src/Summoner/GhcVer.hs
@@ -48,9 +48,11 @@ showGhcVer = \case
     Ghc865  -> "8.6.5"
     Ghc883  -> "8.8.3"
 
--- | These are old GHC versions that are not working with default GHC versions when using Stack.
+{- | These are old GHC versions that are not working with default GHC versions
+when using Stack.
+-}
 oldGhcs :: [GhcVer]
-oldGhcs = [minBound .. Ghc802]
+oldGhcs = [minBound .. Ghc844]
 
 parseGhcVer :: Text -> Maybe GhcVer
 parseGhcVer = inverseMap showGhcVer
@@ -63,7 +65,7 @@ latestLts = \case
     Ghc822  -> "lts-11.22"
     Ghc844  -> "lts-12.26"
     Ghc865  -> "lts-14.27"
-    Ghc883  -> "lts-15.4"
+    Ghc883  -> "lts-15.5"
 
 -- | Represents PVP versioning (4 numbers).
 data Pvp = Pvp

diff --git a/summoner-cli/src/Summoner/Project.hs b/summoner-cli/src/Summoner/Project.hs
@@ -18,7 +18,6 @@ module Summoner.Project
        ) where
 
 import Colourista (bold, formattedMessage, green)
-import Data.List (intersect)
 import NeatInterpolation (text)
 import Relude.Extra.Enum (universe)
 import Shellmet (($?))
@@ -29,7 +28,7 @@ import Summoner.Config (Config, ConfigP (..))
 import Summoner.CustomPrelude (CustomPrelude (..))
 import Summoner.Decision (Decision (..), decisionToBool, decisionsToBools, promptDecisionToBool)
 import Summoner.Default (currentYear, defaultDescription, defaultGHC)
-import Summoner.GhcVer (oldGhcs, parseGhcVer, showGhcVer)
+import Summoner.GhcVer (parseGhcVer, showGhcVer)
 import Summoner.License (LicenseName (..), fetchLicenseCustom, licenseShortDesc, parseLicenseName)
 import Summoner.Mode (ConnectMode (..), Interactivity (..), isOffline)
 import Summoner.Question (YesNoPrompt (..), checkUniqueName, choose, doesExistProjectName,
@@ -123,10 +122,6 @@ generateProjectInteractive connectMode projectName ConfigP{..} = do
         vers -> do
             putTextLn $ "Also these GHC versions will be added: " <> intercalateMap " " showGhcVer vers
             pure vers
-    -- Inform if there are old GHCs that won't be included to Travis Stack matrix
-    let oldGhcIncluded = not $ null $ settingsTestedVersions `intersect` oldGhcs
-    when (oldGhcIncluded && settingsStack && settingsTravis) $
-        warningMessage "Old GHC versions won't be included into Stack matrix at Travis CI because of the Stack issue with newer Cabal versions."
 
     settingsFiles <- fetchSources connectMode cFiles
 

diff --git a/summoner-cli/src/Summoner/Template/GitHub.hs b/summoner-cli/src/Summoner/Template/GitHub.hs
@@ -9,12 +9,13 @@ Maintainer: Kowainik <xrom.xkov@gmail.com>
 This module contains template for GitHub related docs:
 
  * @.gitignore@ — static file with all Haskell related ignored files.
- * @appveyor.yml@ — Appveyor CI for Stack project only.
+ * @.github/workflows/ci.yml@ — GitHub acttions for Cabal projects.
+ * @appveyor.yml@ — Appveyor CI for Cabal or Stack projects.
  * @.travis.yml@ — depending on the build tool and supported GHC versions
    builds the Travis matrix with all necessary checks, including HLint check.
-   __NOTE:__ Old GHC versions is not included into Travis matrix for Stack due to
-   Stack limitations with the Cabal version usage on each GHC. See this issue to
-   track the problem:
+   __NOTE:__ Old GHC versions are included into @allow_failure@ Travis matrix
+   section for Stack due to Stack limitations with the Cabal version usage on
+   each GHC. See this issue to track the problem:
 
     + https://github.com/commercialhaskell/stack/issues/4488
 -}
@@ -23,7 +24,7 @@ module Summoner.Template.GitHub
        ( gitHubFiles
        ) where
 
-import Data.List ((\\))
+import Data.List (delete, intersect)
 import NeatInterpolation (text)
 
 import Summoner.Default (defaultCabal, defaultGHC)
@@ -185,6 +186,7 @@ gitHubFiles Settings{..} = concat
           include:
           $travisCabalMtr
           $travisStackMtr
+          $travisStackAllowFailuresMtr
 
         $installAndScript
 
@@ -207,12 +209,27 @@ gitHubFiles Settings{..} = concat
     travisCabalMatrixItem :: GhcVer -> Text
     travisCabalMatrixItem (showGhcVer -> ghcV) = [text|- ghc: $ghcV|]
 
-    -- Due to Stach issues with newer Cabal versions we are not supporting Travis CI for GHC <= 8.0.2 for stack
+    -- Due to the Stack issues with newer Cabal versions TravisCI for 'oldGhcs'
+    -- can fail. Possible failure jobs are added to the @allow-failures@ section.
     travisStackMtr :: Text
     travisStackMtr = memptyIfFalse settingsStack $ tconcatMap
-           travisStackMatrixItem (settingsTestedVersions \\ (defaultGHC:oldGhcs))
+           travisStackMatrixItem (delete defaultGHC settingsTestedVersions)
         <> travisStackMatrixDefaultItem
 
+    travisStackAllowFailuresMtr :: Text
+    travisStackAllowFailuresMtr = memptyIfFalse (settingsStack && not (null old))
+        [text|
+        $endLine
+        allow_failures:
+        $matrix
+        |]
+      where
+        old :: [GhcVer]
+        old = settingsTestedVersions `intersect` oldGhcs
+
+        matrix :: Text
+        matrix = tconcatMap travisStackMatrixItem old
+
     travisStackMatrixItem :: GhcVer -> Text
     travisStackMatrixItem (showGhcVer -> ghcV) =
         [text|

diff --git a/summoner-cli/test/Test/Show.hs b/summoner-cli/test/Test/Show.hs
@@ -19,5 +19,5 @@ goldenGhcTable =
   , "GHC-8.2.2     base-4.10.1.0   lts-11.22"
   , "GHC-8.4.4     base-4.11.1.0   lts-12.26"
   , "GHC-8.6.5     base-4.12.0.0   lts-14.27"
-  , "GHC-8.8.3     base-4.13.0.0   lts-15.4"
+  , "GHC-8.8.3     base-4.13.0.0   lts-15.5"
   ]