Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support cabal-install 3.0.0.0 in the Makefile #4216

Open
asr opened this issue Nov 20, 2019 · 62 comments
Open

Support cabal-install 3.0.0.0 in the Makefile #4216

asr opened this issue Nov 20, 2019 · 62 comments
Labels
cabal Build problems specifically related to cabal-install devx Developer Experience (IDE setup, linting, dev builds, etc.) (not in changelog) not-in-changelog This issue should not be listed in the changelog. type: enhancement Issues and pull requests about possible improvements
Milestone

Comments

@asr
Copy link
Member

asr commented Nov 20, 2019

I created this issue for collecting/reporting the required changes/issues for supporting cabal-install 3.0.0.0.

@asr asr added type: enhancement Issues and pull requests about possible improvements cabal Build problems specifically related to cabal-install labels Nov 20, 2019
@asr asr added this to the 2.6.1 milestone Nov 20, 2019
@asr asr self-assigned this Nov 20, 2019
@UlfNorell
Copy link
Member

We already made the necessary changes as part of the ghc-8.8 support (#3725) unless I'm very much mistaken. At least I've been using cabal-3.0.0.0 without issue since then. Is the ghc-8.8 Travis job not using 3.0.0.0?

@asr
Copy link
Member Author

asr commented Nov 20, 2019

The Travis job for GHC 8.8.1 is using caba-install 2.4.1.0.

@asr
Copy link
Member Author

asr commented Nov 20, 2019

I got an error when running cabal install in a fresh repository. I'll report the error later.

@L-TChen
Copy link
Member

L-TChen commented Nov 20, 2019

cabal-3.0 works well to me.

The following one is built with GHC 8.8.1 and cabal 3.0

https://travis-ci.org/L-TChen/agda/jobs/614341754

@asr
Copy link
Member Author

asr commented Nov 20, 2019

The above job don't use cabal-install but stack.

@UlfNorell
Copy link
Member

I got an error when running cabal install in a fresh repository.

You have to use cabal v1-install.

@L-TChen
Copy link
Member

L-TChen commented Nov 20, 2019

Stack has not been tested with GHC versions above 8.6, and using 8.8.1, this may fail
Stack has not been tested with Cabal versions above 2.4, but version 3.0.0.0 was found, this may fail

If I was not mistaken, stack uses cabal to install packages ...
https://docs.haskellstack.org/en/stable/faq/#what-is-the-relationship-between-stack-and-cabal

@asr
Copy link
Member Author

asr commented Nov 20, 2019

I think stack uses Cabal the library, but it doesn't use cabal-install the program.

@asr
Copy link
Member Author

asr commented Nov 20, 2019

You have to use cabal v1-install.

I guess. Are you planning to update the user documentation for taking care of the different versions of cabal-install?

@asr
Copy link
Member Author

asr commented Nov 20, 2019

Running cabal-install the first error I got is

$ cabal install
cabal: filepath wildcard 'doc/user-manual.pdf' does not match any files.

This is also the error in Travis when using GHC 8.8.1 and cabal-install 3.0.0.0.

@nad
Copy link
Contributor

nad commented Nov 20, 2019

I think the idea is to eventually deprecate v1-install, so I guess we should at some point make v2-install work.

@ice1000
Copy link
Member

ice1000 commented Nov 24, 2019

I'm building Agda with cabal 3.0.0.0 for weeks. It always work

@L-TChen
Copy link
Member

L-TChen commented Nov 24, 2019

I'm building Agda with cabal 3.0.0.0 for weeks. It always work

Can you make a GH Actions workflow using cabal 3.0 with v2-install? Just make sure that Agda is compatible.

@ice1000
Copy link
Member

ice1000 commented Nov 25, 2019

How did you make the stack one? Is it just porting the .travis.yml?

@L-TChen
Copy link
Member

L-TChen commented Nov 25, 2019

Almost. GHC, cabal (2.0-3.0) and stack are all preinstalled already, so it should be simpler.

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/software-installed-on-github-hosted-runners

@nad
Copy link
Contributor

nad commented Nov 25, 2019

I'm building Agda with cabal 3.0.0.0 for weeks. It always work

Then perhaps the makefile could be updated to use install instead of v1-install. (Just dropping the v1- prefix does not work.)

@L-TChen
Copy link
Member

L-TChen commented Nov 25, 2019

It seems that by removing doc/user-manual.pdf from extra-source-files in Agda.cabal, it actually compiles https://github.com/agda/agda/commit/e956e9d356750c0fe92477c3ecbca70edaf56f2b/checks?check_suite_id=328447539

I haven't run the test suites with the compiled binary, though. Adding this workflow might be a good idea before revising Makefile? I am not able to revise Makefile this year anyway ...

@asr
Copy link
Member Author

asr commented Nov 26, 2019

It seems that by removing doc/user-manual.pdf from extra-source-files in Agda.cabal, it actually compiles.

Yes. Please remove it. I'll add it back in the releases branches.

L-TChen added a commit that referenced this issue Nov 27, 2019
* [ #4216 ] Add the cabal new-build compilation test

* [Actions] Add directory filters to reduce usage; fix travis
@asr asr changed the title Support cabal-install 3.0.0.0 Support cabal-install 3.0.0.0 in the Makefile Dec 13, 2019
@asr asr assigned L-TChen and unassigned asr Dec 13, 2019
@asr asr added the not-in-changelog This issue should not be listed in the changelog. label Dec 13, 2019
@rwe rwe added the devx Developer Experience (IDE setup, linting, dev builds, etc.) (not in changelog) label Aug 31, 2020
@andreasabel andreasabel modified the milestones: icebox, 2.6.3 Jan 27, 2021
@andreasabel
Copy link
Member

To migrate to v2-cabal these problems remain for me:

@mouse07410
Copy link

Could you clarify the status of this issue/effort? How long till there's a version of Makefile (or of mk/cabal.mk) that uses v2- commands?

If one doesn't want to mess with his Haskell environment using cabal v1- commands, how can one build and install the current (master) Agda from the source (cloning this repo), if one wants to also run tests?

Should the following work (in sequence as typed):

  • cabal build | tee >(copy-bins.awk)
  • make bugs
  • make test
    ?

@andreasabel
Copy link
Member

My latest efforts are at:

cabal build --only-dependencies --enable-tests
# Andreas, 2021-09-02:
# Installing Agda, instead of just building it, is to work around
# https://github.com/haskell/cabal/issues/7577
# which hides the build executable from the tests when there is
# a Custom cabal setup (atm works only for Simple setup).
# Once haskell/cabal#7577 is fixed, we can dispose of this workaround.
- name: Install Agda
run: |
mkdir -p ${HOME}/bin
cabal install --installdir=${HOME}/bin
# bin instead of .cabal/bin as this might be more portable (Windows)
- name: Build and test Agda
run: |
export PATH=${HOME}/bin:${PATH}
cabal test

@nad
Copy link
Contributor

nad commented Sep 29, 2021

  • cabal build | tee >(copy-bins.awk)

If you have a recent version of cabal-install, then you can perhaps use cabal list-bin to find the path to the executable.

@mouse07410
Copy link

$ cabal list-bin agda
cabal: No or multiple targets given

$ cabal list-bin agda-tests
cabal: No or multiple targets given

$ cabal list-bin agda-mode
cabal: No or multiple targets given

$ cabal list-bin
cabal: One target is required, none provided

$ cabal list-bin --project-file=Agda.cabal agda
cabal: There is no <pkgname>.cabal package file or cabal.project file. To
build packages locally you need at minimum a <pkgname>.cabal file. You can use
'cabal init' to create one.

For non-trivial projects you will also want a cabal.project file in the root
directory of your project. This file lists the packages in your project and
all other build configuration. See the Cabal user guide for full details.

$ cabal list-bin --project-file=Agda.cabal agda-mode
cabal: There is no <pkgname>.cabal package file or cabal.project file. To
build packages locally you need at minimum a <pkgname>.cabal file. You can use
'cabal init' to create one.

For non-trivial projects you will also want a cabal.project file in the root
directory of your project. This file lists the packages in your project and
all other build configuration. See the Cabal user guide for full details.

@mouse07410
Copy link

Also, please take a look at haskell/cabal#7693 - perhaps it would be helpful here, or you might be able to give some useful feedback there.

@fgaz
Copy link

fgaz commented Sep 29, 2021

#4216 (comment) looks like haskell/cabal#7679

@mouse07410
Copy link

#4216 (comment) looks like haskell/cabal#7679

They are two different aspects of the same problem:

@mouse07410
Copy link

As asked here - what's the point of having --only-dependencies in the install command? Since we install the generated binaries before running tests anyway - why not ditch that parameter altogether???

@andreasabel
Copy link
Member

As asked here - what's the point of having --only-dependencies in the install command? Since we install the generated binaries before running tests anyway - why not ditch that parameter altogether???

Dunno, maybe @asr can say more.

I can see a use for --only-dependencies in the following cases:

  • CI: First install dependencies, cache them for the next time, then install Agda but don't cache it. (This is what is often done in the github workflows.)
  • Developer: Install dependencies with optimization on, but build Agda without optimizations for quick testing. (This at least made sense with v1-cabal, the situation with v2-cabal is more confusing.)

@mouse07410
Copy link

  1. CI: I don't think dependencies that did not change themselves, would be rebuilt - therefore, no need to worry about them.
  2. Developer: I understand the logic, but I'm pretty sure that (unlike in C/C++) you cannot build dependencies with one set of flags, and then build the "main" package with another set of flags without having all the dependencies rebuilt. Difference in the flag set can be minimal - one flag would be sufficient to force the rebuild/recompile.

@L-TChen
Copy link
Member

L-TChen commented Sep 30, 2021

  • CI: First install dependencies, cache them for the next time, then install Agda but don't cache it. (This is what is often done in the github workflows.)

As for CI, the main benefit of using --only-dependencies, I think, is to navigate the error information easier.

@mouse07410
Copy link

mouse07410 commented Sep 30, 2021

OK, so having had all this discussion and observing what Cabal maintainers got to say about this, where are we regarding move to cabal v2- build?

Update

It looks like the sequence for cabal v2-... should be

  1. cabal v2-build --enable-tests --only-dependencies (optional, I don't fully understand what separating this step accomplishes, given that CI probably builds everything)
  2. cabal v2-build --enable-tests
  3. actually running tests - and here's my stumbling block: how do I get all the Agda tests to run, given that I know where agda, agda-mode, and agda-tests binaries are?
  4. finally, cabal v2-install that will place the Agda binaries into the global store ~/.cabal/bin and ~/.cabal/store

Do you concur? If so, how to deal with (3) above?

Update 2

Specifically, what scripts and mk/*.mk files do I need to modify, to get make test working with this new build? Assuming that I know the locations of agda, agda-mode, and agda-tests?

andreasabel added a commit that referenced this issue Jan 30, 2022
Uses a dedicated `cabal.project` file to pass the necessary
`ghc-options` that skip code generation.
andreasabel added a commit that referenced this issue Mar 15, 2022
Uses a dedicated `cabal.project` file to pass the necessary
`ghc-options` that skip code generation.
andreasabel added a commit that referenced this issue Mar 16, 2022
Uses a dedicated `cabal.project` file to pass the necessary
`ghc-options` that skip code generation.
@jespercockx jespercockx modified the milestones: 2.6.3, later Aug 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cabal Build problems specifically related to cabal-install devx Developer Experience (IDE setup, linting, dev builds, etc.) (not in changelog) not-in-changelog This issue should not be listed in the changelog. type: enhancement Issues and pull requests about possible improvements
Projects
None yet
Development

No branches or pull requests

10 participants