[question] Conan uses home directory for cmake cache #17434
Comments
|
Thanks for your report. Lets start with some preliminary test, remove that CMakeCache.txt, lets try to define a new Conan cache folder with: And share the full output and if it happened to create the CMakeCache.txt again. |
|
Hey, when I do that I get an SSL error to the center2.conan.io. Here is the full output I had the same issue with my regular home, so knowing this is not the issue for my original post, i went ahead and changed center2.conan.io to center.conan.io in the remotes.json. this fixes the ssl issue and the build No CMakeCache was created in my home dir, which leads me to believe that some of my settings are at fault here, I am just trying to figure out which settings could result in the observed behaviour |
This could be due to some network constraints, are you using a corporate VPN, ZScaler, corporate proxies or the like? In that case, you might need to request your IT to enable
Yes, I see that you have some weird output in your logs, I am not sure if your profile was incorrect or something but this looked wrong:
|
Alright, I will open a ticket with our IT for that.
What exactly do you think looks wrong here? the settings section is equal to what a profile detect returned, except for the cppstd is bumped to 20, as we use C++20 in our current project I took the two conf settings from The powershell option is self explanatory, powershell is just a much more convenient shell then cmd. |
|
It was missing line endings, so maybe some things were not being parsed correctly |
|
Interesting. Are profiles expected to have unix or windows line endings? It is a bit hard to keep them consistent, as gitattributes usually match line endings to text files. Edit: changing all line endings to linux line endings did not change the result, the issue still persists Edit2: The line ending display issue seems to come from github. In the post "source" the newlines are present |
|
After some attempts the culprit seems to be the line in my global.conf. This is a bummer, as this used to work fine in previous versions. It would be great if the run-/buildenvironments would work with powershell. Thank you very much for your help on the issue @memsharded . What do you recommend are the next steps? The option is an officially documented one but it seems to result in broken behaviour. |
Recommended to use Unix LF only everywhere, see:
All conan generated files are LF only
But this still doesn't explain why the CMakeCache.txt ends there, it is expected that it is possible to generate .ps1 files for the environment too. So it would be great to reduce as much the example and try to come up with something that is reproducible, and we can reproduce on our side, if it is a bug, so we can fix it. |
|
I expanded the minimum reproducible test and can confirm that I get the same error with just setting As expected I get the same issue, here the output of the conan install command. This time I added -vtrace, maybe that helps a bit Note that this mentions the last build made from the default .conan2 directory, as the CMakeCache in my user home still contained the CMakeCache.txt of the last "regular" run (without CONAN_HOME set) Edit: Because I never mentioned it, my OS is Windows 10 x64 Enterprise with update 22H2 |
|
I fired up one of our Test VMS that never had conan installed and was unable to reproduce the issue. I installed the same version of conan, changed the global.conf but the conan install step was successful and no CMakeCache was written into weird places. Is there any other Environment variable or similar that could affect conans behaviour? |
|
Yes, in Windows systems, the logic to obtain the user home folder (like if path[0] != '~':
return path
# In win these variables should exist and point to user directory, which
# must exist.
home = os.environ.get("HOME")
try:
# Problematic cases of wrong HOME variable
# - HOME = %USERPROFILE% verbatim, as messed by some other tools
# - MSYS console, that defines a different user home in /c/mingw/msys/users/xxx
# In these cases, it is safe to remove it and rely on USERPROFILE directly
if home and (not os.path.exists(home) or
(os.getenv("MSYSTEM") and os.getenv("USERPROFILE"))):
del os.environ["HOME"]
result = os.path.expanduser(path)
finally:
if home is not None:
os.environ["HOME"] = home
return resultAs you can see it can depend on HOME, MSYSTEM and USERPROFILE. It is possible that you have some configuration in your dev machine that is different to the Test VMS. |
|
I created this quick python script that tests all the values used in the snippet and it's output. import os
import sys
print(os.environ.get("HOME"))
print(str(os.path.exists(os.environ.get("HOME"))))
print(str(os.getenv("MSYSTEM")))
print(str(os.getenv("USERPROFILE")))
print(str(os.environ["HOME"]))
print(str(os.path.expanduser(sys.argv[0]))) # <<< I assume this is what "path" is in the snippetI did not see anything wrong here, however, when i test the same snippet on the VM, i get the following output: I check environment variables and yes, my real machine had a "HOME" environment variable defined while my VM did not have that variable. This seems to be the cause for the issue. Now I do not know why my main pc has this variable defined, I would assume (as you said aswell) that "HOME" is a valid enviroment variable, but it turns out this is not a standard windows environment variable but must have been set by some application I installed in the past. This got me excited, I would have hoped that this fixed the issue, so I removed the environment entry of HOME, started a fresh terminal.... but i was letdown when the conan output did not change. I still get the same output, even thought the python snippet now returns the same exception as on the VM. I adjusted the python snippet a bit to disable the exception throwing parts and now get the same output on both my local PC and my VM import os
import sys
print(os.environ.get("HOME"))
#print(str(os.path.exists(os.environ.get("HOME"))))
print(str(os.getenv("MSYSTEM")))
print(os.getenv("USERPROFILE"))
#print(str(os.environ["HOME"]))
print(str(os.path.expanduser(os.path.abspath(sys.argv[0])))) # <<< I assume this is what "path" is in the snippet?Are there any other configs, settings etc. that could interfere here? |
|
Is it possible that the Conan app is running in some other virtualenv or context that might have defined or retained the old env-var values? You can print Another possibility is to go to your Conan installation (if it is a Python installation), look for the |
|
First off I want to thank you again for all the work you put into my local issue here. The conan version gives me the following output This is a bit weird, as Adding I have conan installed via chocolatey ( I found no conan/internal/paths.py, so I removed my choco installation and reinstalled conan 2.10.1 via pip and I now get the The error still persist, even with the pip installation. I went into def _conan_expand_user(path):
""" wrapper to the original expanduser function, to workaround python returning
verbatim %USERPROFILE% when some other app (git for windows) sets HOME envvar
"""
path = str(path)
print("_conan_expand_user: path " + path)
if path[0] != '~':
print("_conan_expand_user: if path[0] != '~' true")
return path
print("_conan_expand_user: if path[0] != '~' false")
# In win these variables should exist and point to user directory, which
# must exist.
home = os.environ.get("HOME")
print("_conan_expand_user: home " + str(home))
try:
# Problematic cases of wrong HOME variable
# - HOME = %USERPROFILE% verbatim, as messed by some other tools
# - MSYS console, that defines a different user home in /c/mingw/msys/users/xxx
# In these cases, it is safe to remove it and rely on USERPROFILE directly
#THESE THROW EXCEPTIONS AND THEREFORE ARE COMMENTED
#print("_conan_expand_user: not os.path.exists(home) " + str(not os.path.exists(home)))
#print("_conan_expand_user: os.getenv('MSYSTEM') " + str(os.getenv("MSYSTEM")))
#print("_conan_expand_user: os.getenv('USERPROFILE') " + str(os.getenv("USERPROFILE")))
if home and (not os.path.exists(home) or
(os.getenv("MSYSTEM") and os.getenv("USERPROFILE"))):
print("_conan_expand_user: deleting 'HOME' envvar ")
del os.environ["HOME"]
result = os.path.expanduser(path)
print("_conan_expand_user: os.path.expanduser(path) " + str(os.path.expanduser(path)))
finally:
print("_conan_expand_user: entered finally block ")
if home is not None:
print("_conan_expand_user: set envvar HOME to home " + str(home))
os.environ["HOME"] = home
print("_conan_expand_user: gracefully reached return statement " + str(result))
return resultHere is the relevant output in my failing conan install step: This looks all correct to me. Edit: I also tried going through the other conan .py files in 'C:\Python311\Lib\site-packages\conan' trying to find how setting "tools.env.virtualenv:powershell" would change anything but that was like looking for a needle in a haystack. I then tried installing older and older versions of conan to see if the issue was introduced in a specific version but after going all the way back to 2.1 the issue still persistet throughout all versions, so I doubt that this is an issue introduced by conan and probably some issue intorduce by my own environment (even though I have no clue what could be causing this) Edit2: just to rule it out i tried to move the |
Exactly, this means that you are using the Conan Windows installer, not the Python I don't know what else to check, I am running out of ideas... Can you please retry, just |
|
Hey @memsharded , thank you again for your in depth help with the issue. I fear believe that the issue does not lie with conan, so I am going to investigate a bit on my own, I will give feedback if I managed to discover something new, if not feel free to close this issue after it gets stale. for the conan install command you wrote, this is the output: I believe the issue comes from the powershell.exe call. Whenever I am inside the terminal in any folder and I call "powershell.exe" the new powershell instance starts up in my home folder. This is different to the behaviour I found described online and also different to how powershell behaves in my VM, where it starts in the current folder On my machine: On my VM: This is definitely the issue that happens when conan calls Powershell in |
|
I FIGURED IT OUT! I checked if any Powershell profiles were installed by checking all possible locations using This profile was changing the working directory of all powershell instances running under my user to my home directory. THANK YOU VERY MUCH @memsharded FOR YOUR HELP AND INVESTIGATION I AM SO SORRY FOR CAUSING SO MUCH TROUBLE. Apparently creating such a profile is a (popular ?) way to make powershell start in specific directory, at least a lot of google results talk about this when looking for "powershell start in working directory" (for example here) |
|
The powershell has a parameter We might need the expertise of other users like @Todiq |
|
Hello, Thanks for the ping, I am happy if I can help. I can only speak about my experience: I used to have oh-my-posh installed and had a starting directory in my profile as well. However, that caused a lot of troubles, especially with VS Code. In the end I ditched everything and simply set the starting directory in the My best guess is that many people would have a profile edited for their specific needs and that starting powershell with Maybe adding a conf to ignore the profile could mitigate this kind of issues? |
|
#17416 has added support via |
What is your question?
This might be an odd question, I have a weird issue with conan and I am out of ideas how this might have happened
In short, all conan recipes fail to build, because the CMakeCache.txt is written to my home directory. Not somewhere in the .conan subdirectory in my home, no, direcly into my home dir
C:\Users\<GulaschLulatsch>\CMakeCache.txt.I am at a loss to why this would happen and it took me quite a while to figure out that this is the reason my builds failed.
This initially happened while I was on conan 2.9.3 but in my attempts to fix the issue i have upgraded to the latest conan 2.10.1 and the issue still persists.
Here is an output of one failed install. I pasted the whole log but only the end is really important: the build directory is reported correctly but the build fails, because the CMakeCache contains output of a previous conan build step.
Conan output
======== Input profiles ======== Profile host: [settings] arch=x86_64 build_type=Release compiler=msvc compiler.cppstd=20 compiler.runtime=dynamic compiler.runtime_type=Release compiler.version=194 os=Windows [conf] tools.env.virtualenv:powershell=True tools.microsoft.msbuild:vs_version=17Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=20
compiler.runtime=dynamic
compiler.runtime_type=Release
compiler.version=194
os=Windows
[conf]
tools.env.virtualenv:powershell=True
tools.microsoft.msbuild:vs_version=17
======== Computing dependency graph ========
aws-sdk-cpp/1.11.352: Checking remote: MedConanServer
aws-crt-cpp/0.26.9: Checking remote: MedConanServer
aws-c-common/0.9.15: Checking remote: MedConanServer
aws-c-sdkutils/0.1.15: Checking remote: MedConanServer
aws-c-io/0.14.7: Checking remote: MedConanServer
aws-c-cal/0.6.14: Checking remote: MedConanServer
aws-c-compression/0.2.18: Checking remote: MedConanServer
aws-c-http/0.8.1: Checking remote: MedConanServer
aws-c-auth/0.7.16: Checking remote: MedConanServer
aws-c-mqtt/0.10.3: Checking remote: MedConanServer
aws-checksums/0.1.18: Checking remote: MedConanServer
aws-c-event-stream/0.4.2: Checking remote: MedConanServer
aws-c-s3/0.5.5: Checking remote: MedConanServer
zlib/1.3.1: Checking remote: MedConanServer
cmake/3.30.5: Checking remote: MedConanServer
Graph root
conanfile.py (tmdgateway/0.0.0.4@development): C:\Users\fabiang\source\repos\tmd_gateway\conanfile.py
Requirements
aws-c-auth/0.7.16#d1c779b29e87d9b9e6564a1b4a5a2e28 - Cache (MedConanServer)
aws-c-cal/0.6.14#3ed4cc4d6b5ebe63bcbed6fd0fa9951f - Cache (MedConanServer)
aws-c-common/0.9.15#c44a2ce5f1e858caf1a5b692078755eb - Cache (MedConanServer)
aws-c-compression/0.2.18#c86b8f00cee8d572cd71835e9842405f - Cache (MedConanServer)
aws-c-event-stream/0.4.2#a390c1c504451a7eeab447bfbb4a9e47 - Cache (MedConanServer)
aws-c-http/0.8.1#2dbdd363d5daf73ead6877310e406ebe - Cache (MedConanServer)
aws-c-io/0.14.7#3ac4e4b9a0d26e4d22592076f8f06171 - Cache (MedConanServer)
aws-c-mqtt/0.10.3#a4d43fcbcc1e0a91079ec93691dd64d2 - Cache (MedConanServer)
aws-c-s3/0.5.5#cae5e2180e586fe5c10a278999a7ca8b - Cache (MedConanServer)
aws-c-sdkutils/0.1.15#6cfaa1f582765f5fabf9609f7626522c - Cache (MedConanServer)
aws-checksums/0.1.18#7039b79391bd012855848816cb663464 - Cache (MedConanServer)
aws-crt-cpp/0.26.9#a63cd7ec7a0b15cfa417cecc4a9cf6e9 - Cache (MedConanServer)
aws-sdk-cpp/1.11.352#f1ee18b381833da356f2e023f34f0d63 - Cache (MedConanServer)
zlib/1.3.1#f52e03ae3d251dec704634230cd806a2 - Cache (MedConanServer)
Build requirements
cmake/3.30.5#540068cc37f1d21b42b729c82d6bcb29 - Cache (MedConanServer)
Resolved version ranges
cmake/[>=3.28 <4]: cmake/3.30.5
zlib/[>=1.2.11 <2]: zlib/1.3.1
======== Computing necessary packages ========
aws-c-common/0.9.15: Forced build from source
cmake/3.30.5: Forced build from source
zlib/1.3.1: Forced build from source
aws-c-cal/0.6.14: Forced build from source
aws-c-compression/0.2.18: Forced build from source
aws-c-sdkutils/0.1.15: Forced build from source
aws-checksums/0.1.18: Forced build from source
aws-c-io/0.14.7: Forced build from source
aws-c-event-stream/0.4.2: Forced build from source
aws-c-http/0.8.1: Forced build from source
aws-c-auth/0.7.16: Forced build from source
aws-c-mqtt/0.10.3: Forced build from source
aws-c-s3/0.5.5: Forced build from source
aws-crt-cpp/0.26.9: Forced build from source
aws-sdk-cpp/1.11.352: Forced build from source
Requirements
aws-c-auth/0.7.16#d1c779b29e87d9b9e6564a1b4a5a2e28:58346eae9e30a7f827e6fb4668ca5de1764be1e6 - Build
aws-c-cal/0.6.14#3ed4cc4d6b5ebe63bcbed6fd0fa9951f:6b980ee21a389d682592bb927a3ee9bade2744ca - Build
aws-c-common/0.9.15#c44a2ce5f1e858caf1a5b692078755eb:f815149021754f2756c6103a8a3d7ce03d91d016 - Build
aws-c-compression/0.2.18#c86b8f00cee8d572cd71835e9842405f:6b980ee21a389d682592bb927a3ee9bade2744ca - Build
aws-c-event-stream/0.4.2#a390c1c504451a7eeab447bfbb4a9e47:d7a3b9a3f3b673e4c597187cd7da533744380316 - Build
aws-c-http/0.8.1#2dbdd363d5daf73ead6877310e406ebe:8c67cb283fd7116704f3964ecf01d99b881619dc - Build
aws-c-io/0.14.7#3ac4e4b9a0d26e4d22592076f8f06171:ebb944089878ae881a64e6e71101931644f478f9 - Build
aws-c-mqtt/0.10.3#a4d43fcbcc1e0a91079ec93691dd64d2:82ca430ec863b14cc5edb2e142d11cac94baa3fb - Build
aws-c-s3/0.5.5#cae5e2180e586fe5c10a278999a7ca8b:1631aef96833429ad5b1c10d1e4a7650a4aa87e1 - Build
aws-c-sdkutils/0.1.15#6cfaa1f582765f5fabf9609f7626522c:6b980ee21a389d682592bb927a3ee9bade2744ca - Build
aws-checksums/0.1.18#7039b79391bd012855848816cb663464:6b980ee21a389d682592bb927a3ee9bade2744ca - Build
aws-crt-cpp/0.26.9#a63cd7ec7a0b15cfa417cecc4a9cf6e9:4a6175057d987be596dcebda74c44c4065ab6571 - Build
aws-sdk-cpp/1.11.352#f1ee18b381833da356f2e023f34f0d63:6053bb50bbd6340bc98017fb36779311e33c7142 - Build
zlib/1.3.1#f52e03ae3d251dec704634230cd806a2:0d6dd492a7d31822b2f2686ec67bbaef586416a3 - Build
Build requirements
cmake/3.30.5#540068cc37f1d21b42b729c82d6bcb29:522dcea5982a3f8a5b624c16477e47195da2f84f - Build
======== Installing packages ========
-------- Installing package aws-c-common/0.9.15 (1 of 15) --------
aws-c-common/0.9.15: Building from source
aws-c-common/0.9.15: Package aws-c-common/0.9.15:f815149021754f2756c6103a8a3d7ce03d91d016
aws-c-common/0.9.15: Copying sources to build folder
aws-c-common/0.9.15: Building your package in C:\Users\fabiang.conan2\p\b\aws-cdf64e0ba5f789\b
aws-c-common/0.9.15: Calling generate()
aws-c-common/0.9.15: Generators folder: C:\Users\fabiang.conan2\p\b\aws-cdf64e0ba5f789\b\build\generators
aws-c-common/0.9.15: CMakeToolchain generated: conan_toolchain.cmake
aws-c-common/0.9.15: CMakeToolchain generated: C:\Users\fabiang.conan2\p\b\aws-cdf64e0ba5f789\b\build\generators\CMakePresets.json
aws-c-common/0.9.15: CMakeToolchain generated: C:\Users\fabiang.conan2\p\b\aws-cdf64e0ba5f789\b\src\CMakeUserPresets.json
aws-c-common/0.9.15: Generating aggregated env files
aws-c-common/0.9.15: Generated aggregated env files: ['conanbuild.ps1', 'conanrun.ps1']
aws-c-common/0.9.15: Calling build()
aws-c-common/0.9.15: apply_conandata_patches(): No patches defined in conandata
aws-c-common/0.9.15: Running CMake.configure()
aws-c-common/0.9.15: RUN: cmake -G "Visual Studio 17 2022" -DCMAKE_TOOLCHAIN_FILE="generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="C:/Users/fabiang/.conan2/p/b/aws-cdf64e0ba5f789/p" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" "C:/Users/fabiang/.conan2/p/b/aws-cdf64e0ba5f789/b/src"
CMake Error: The source "C:/Users/fabiang/.conan2/p/b/aws-cdf64e0ba5f789/b/src/CMakeLists.txt" does not match the source "C:/Users/fabiang/.conan2/p/b/aws-c7524aa71d13e0/b/src/CMakeLists.txt" used to generate cache. Re-run cmake with a different source directory.
aws-c-common/0.9.15: ERROR:
Package 'f815149021754f2756c6103a8a3d7ce03d91d016' build failed
aws-c-common/0.9.15: WARN: Build folder C:\Users\fabiang.conan2\p\b\aws-cdf64e0ba5f789\b\build
ERROR: aws-c-common/0.9.15: Error in build() method, line 74
cmake.configure()
ConanException: Error 1 while executing
First I suspected my profile could be interfering so I deleted my profiles, but the issue still persist with the default profile generated by
conan profile detect.Restarting the PC did not change anything, maybe some weird background routine could interfere, but no change.
I suspected maybe my global.conf could be responsible for this but it looks unconspicuous to me
global.conf
Core configuration (type 'conan config list' to list possible values)
e.g, for CI systems, to raise if user input would block
core:non_interactive = True
some tools.xxx config also possible, though generally better in profiles
tools.android:ndk_path = my/path/to/android/ndk
core:default_build_profile = default
core:default_profile = default
tools.env.virtualenv:powershell = True
Then I suspected maybe there is some weird setting in my conanfile so i tried to simple
conan install --ref="gtest/1.15.0"--build="*"` yet still i get the same error and conan writing to my home.What else could be the root of this issue? I also have the official conan2 extensions installed from here https://github.com/conan-io/conan-extensions but I cannot see how i can quickly uninstall them
Have you read the CONTRIBUTING guide?
The text was updated successfully, but these errors were encountered: