Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Add RelocPath type for userdefined relocatable paths #56053

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

fatteneder
Copy link
Member

RelocPath(path) works by splitting path as depot/subpath based on DEPOT_PATH during construction.

When using RelocPath one explicitly opts into relocation, so it throws when path cannot be split.

A relocated path can be queried with String(r::RelocPath), which also throws on failure.


Fixes #54430
Alternative to #55146

@fatteneder fatteneder added the compiler:precompilation Precompilation of modules label Oct 8, 2024
base/loading.jl Outdated Show resolved Hide resolved
@topolarity
Copy link
Member

What's the idiomatic usage pattern for this look like?

@fatteneder
Copy link
Member Author

The example where @__RELOCDIR__ fails #55146 (comment) would become now:

module MyPkg

const data = RelocPath(joinpath(@__DIR__, "myfile.data"))

function read_data()
    path = String(data)
    # load from path ...
end

end

@gbaraldi
Copy link
Member

gbaraldi commented Oct 9, 2024

Is this how we get a Path type finally. I'd love that

@DilumAluthge DilumAluthge requested a review from gbaraldi October 17, 2024 23:26
@DilumAluthge
Copy link
Member

It would be nice to get this into Julia 1.12.

base/loading.jl Outdated Show resolved Hide resolved
base/loading.jl Outdated Show resolved Hide resolved
@DilumAluthge
Copy link
Member

Should we have a way of constructing a RelocPath by just providing the subpath?

Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
@fatteneder
Copy link
Member Author

Is this how we get a Path type finally. I'd love that

FWIW there is already https://juliahub.com/ui/Packages/General/FilePathsBase and https://juliahub.com/ui/Packages/General/FilePaths which provide such types.

Should we have a way of constructing a RelocPath by just providing the subpath?

I am not sure.

@DilumAluthge
Copy link
Member

@vchuravy @gbaraldi and @topolarity Would you be able to review this PR?

As far as I can tell, this PR is an improvement compared to #55146, but I'm not particularly familiar with the relevant part of the codebase.

@DilumAluthge
Copy link
Member

@staticfloat IIRC, you also worked on the depot-relocatibility code; it would be great if you could review this PR.

@DilumAluthge
Copy link
Member

Is this how we get a Path type finally. I'd love that

The RelocPath constructor will error if the path isn't inside a depot, right? So IIUC this doesn't really cover the general "path type" request.

Comment on lines +3375 to +3376
struct RelocPath
subpath::String
Copy link
Member

@vtjnash vtjnash Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts about using OncePerProcess to make it so we only compute the result once:

Suggested change
struct RelocPath
subpath::String
struct RelocPath
getpath::OncePerProcess{String}
end
RelocPath(path::AbstractString) = let
depot, _ = replace_depot_path_impl(path)
if isnothing(depot)
error("Failed to locate $(path) in any of DEPOT_PATH.")
end
subpath = String(replace(path, depot => ""; count=1))
return RelocPath(OncePerProcess{String}() do
for d in DEPOT_PATH
if isdirpath(d)
d = dirname(d)
end
local path = string(d, subpath)
if ispath(path)
return path
end
end
error("Failed to relocate @depot$(subpath) in any of DEPOT_PATH.")
end)
end
String(r::RelocPath) = r.getpath() # dynamically dispatches since getpath isn 't concrete :/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love this. What if someone has modified Base.DEPOT_PATH during their Julia session?

Comment on lines +3400 to +3402
function show(io::IO, r::RelocPath)
print(io, string("RelocPath(\"@depot", r.subpath, "\")"))
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty inaccurate, since it doesn't support @depot

Suggested change
function show(io::IO, r::RelocPath)
print(io, string("RelocPath(\"@depot", r.subpath, "\")"))
end

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we allow RelocPath to accept paths that are literally @depot/foo/bar?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as far as the original show method goes, I think that the meaning of @depot/ is reasonably easy to understand here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty inaccurate, since it doesn't support @depot

But it does behave like a @depot. (it behaves more like include_dependency than include to be precise)

What is inaccurate is that show assumes that nobody tinkered with DEPOT_PATH since construction.

@topolarity
Copy link
Member

Should we just call this DepotPath? I think that might be a clearer name for users

@fatteneder
Copy link
Member Author

Should we just call this DepotPath? I think that might be a clearer name for users

No strong feelings from my side.

@DilumAluthge
Copy link
Member

I also don't feel strongly between RelocPath and DepotPath.

We could also do RelocatablePath if we wanted something a little more unambiguous.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:precompilation Precompilation of modules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Some uses of @__DIR__ should be relocatable.
5 participants