Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify the behavior of file! #134442

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Specify the behavior of file! #134442

wants to merge 1 commit into from

Conversation

epage
Copy link
Contributor

@epage epage commented Dec 17, 2024

This takes the current behavior of file! and documents it so it is safe to make assumptions about.
For example, Cargo could provide a CARGO_RUSTC_CURRENT_DIR as a base path for file!.

Example use cases

  • Being able to look up test assets relative to the current file (example)
  • Inline snapshotting libraries being able to update Rust source code (example)

See rust-lang/cargo#3946 for more context.

T-libs-api discussed two solutions in rust-lang/libs-team#478

  • file_absolute!:
    • Has less meaning in other build tools like buck2
    • Bakes in the assumption that a full path is available (e.g. with trim-paths)
  • Specifying file!s behavior (this PR):
    • Leaves it to the user to deal with trim-paths
    • Even though file! is currently unspecified, changing it would likely have too large of an impact on the ecosystem at this time.

A future possibility is that rustc could have a flag that controls modifies the base path used for file!.
That seems purely additive with specifying the behavior and we do not want to block on it.
It would also likely be too disruptive for Cargo users (as mentioned). However, we tried to keep this in mind when specifying the behavior.

This takes the current behavior of `file!` and documents it so it is
safe to make assumptions about.
For example, Cargo could provide a `CARGO_RUSTC_CURRENT_DIR` as a base
path for `file!`.

Example use cases
- Being able to look up test assets relative to the current file
  ([example](https://github.com/rust-lang/cargo/blob/b9026bf654d7fac283465e58b8b76742244ef07d/tests/testsuite/cargo_add/add_basic/mod.rs#L34))
- Inline snapshotting libraries being able to update Rust source code
  ([example](https://github.com/rust-lang/cargo/blob/b9026bf654d7fac283465e58b8b76742244ef07d/tests/testsuite/alt_registry.rs#L36-L45))

See rust-lang/cargo#3946 for more context.

T-libs-api discussed two solutions in rust-lang/libs-team#478
- `file_absolute!`:
  - Has less meaning in other build tools like buck2
  - Bakes in the assumption that a full path is available (e.g. with
    trim-paths)
- Specifying `file!`s behavior (this PR):
  - Leaves it to the user to deal with trim-paths
  - Even though `file!` is currently unspecified, changing it would
    likely have too large of an impact on the ecosystem at this time.

A future possibility is that rustc could have a flag that controls
modifies the base path used for `file!`.
That seems purely additive with specifying the behavior and we do not
want to block on it.
It would also likely be too disruptive for Cargo users (as mentioned).
However, we tried to keep this in mind when specifying the behavior.
@epage epage added the T-lang Relevant to the language team, which will review and decide on the PR/issue. label Dec 17, 2024
@rustbot
Copy link
Collaborator

rustbot commented Dec 17, 2024

r? @workingjubilee

rustbot has assigned @workingjubilee.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 17, 2024
@epage
Copy link
Contributor Author

epage commented Dec 17, 2024

I assume @rust-lang/libs-api would want to review the text but, from what I understand, T-lang is the ultimate decider on built-in macros.

@joshtriplett
Copy link
Member

This documents the existing behavior, and it is hedging about potential future changes (compiler command-line options). Nonetheless, since this could be interpreted as a new guarantee, it's a potential one-way door, so:

@rfcbot merge

Nominating for T-lang discussion.

@rfcbot
Copy link

rfcbot commented Dec 17, 2024

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

Concerns:

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

@rfcbot rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Dec 17, 2024
@joshtriplett joshtriplett added I-lang-nominated Nominated for discussion during a lang team meeting. I-lang-easy-decision Issue: The decision needed by the team is conjectured to be easy; this does not imply nomination proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Dec 17, 2024
@nikomatsakis
Copy link
Contributor

@rfcbot reviewed

1 similar comment
@traviscross
Copy link
Contributor

@rfcbot reviewed

@rfcbot rfcbot added final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. labels Dec 18, 2024
@rfcbot
Copy link

rfcbot commented Dec 18, 2024

🔔 This is now entering its final comment period, as per the review above. 🔔

Comment on lines 1298 to 1299
/// The file name is derived from the source path passed to the Rust compiler and any path operations
/// performed on top of that (e.g. `#[path = "<child>"]` is joined to the current `file!`),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems confusing. You mean in the "child" module in question? Isn't that just, necessarily, the source path "passed to the Rust compiler"?

It seems like it's trying to mention this example as if it's an exception, when in reality "finding the path to an out-of-line module" is always a "path operation".

Copy link
Member

@dtolnay dtolnay Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This made sense to me as written. The filepath passed to the Rust compiler is always only the path of the root module. In every other module, the value of file! is the result of a sequence of well-specifiable path operations performed on the root module's path.

For example in rustc path/to/root.rs containing a #[path = "some/where.rs"] module, file! in the submodule would be "path/to/some/where.rs".

This paragraph is indicating that "path/to/some/where.rs" is derived from (sensitive to) 2 things:

  1. The root module's source path as passed to the Rust compiler. So if you had run rustc /absolute/path/to/root.rs instead, or rustc ./wat/../path/to/root.rs, you should expect that the submodule's file! would be different, even though neither the root module nor child module has changed location in the filesystem. Ed's documentation is pointing out the important fact that the path passed to the compiler is relevant.

  2. The sequence through which we arrive from the root module to the module containing the file! call site, whether that's mod or #[path = …] or include!(…). If you change that, you should expect that file! can return something different in the child module.

From your comment, it is not clear to me what possible unintended interpretation of the documentation is taking place. Is it about whether #[path = "<child>"] constitutes a "path passed to the Rust compiler"? (It does not. The path passed to the Rust compiler refers to a command-line argument.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's confusing to an expert, then perhaps we might benefit by saying more or saying it differently. In fact, much of what you wrote by way of explanation could itself make sense in the documentation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. #[path] usually is an exceptional case (because it is designed to allow exceptional cases). mentioning an exceptional case without describing the normal case can mislead people in a number of directions. this includes the situation when the exceptional case is functionally identical to the normal case.
  2. "what operations the compiler performs on filesystem paths such that I can understand what file! will give the name of" is not really addressed by the documentation at hand. it seems about 200 additional words was needed to describe what those are?
  3. the purpose of our public-facing API documentation is to be useful to people who are not familiar with the vagaries of the matter at hand.
  4. we already have documentation that better describes the way files are discovered on the filesystem, and are probably better served by relating things back to it and expanding it if necessary: https://doc.rust-lang.org/reference/items/modules.html#module-source-filenames
  5. @dtolnay you seem to have noticed that remarks that are overly terse are hard to interpret. are you sure you were correct in interpreting "This seems confusing" as "I am confused"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if it was confusing, let's improve it. However, I don't see dtolnay's comments saying we shouldn't but providing an anchor for the conversation to find what aspect was confusing (and it was nice to get confirmation that I wasn't so far off the mark as a contributor). Thats important for understanding how to fix it. I find it helpful because I'm not fully clear on what the initial feedback is (it makes it sound like I'm saying <child> is the root source module?).

So going off of the more specific points and with dtolnay's wording, I've made some tweaks. I hope they helped. In giving feedback, please help me understand why it seems confusing so we can discuss points on it.

I chose #[path] because I felt it was simpler to provide an example (mod file vs non-mod, directory vs file mod, etc) but has some confusing parts as well (still conditioned on mod-file vs non-mod. I found making the example more concrete can bypass those cases and switched to using mod foo;.

I hadn't thought of delegating to the reference but in looking at the reference, I worry that doing so would also be confusing because its framed around answering a different question (the module system) with assumptions like paths are relative to the source module's directory. I could see possibly linking to it for more details.

Something I want to be cautious about is distracting from the core messages of this documentation. The core point of file! is whats currently documented. The core message of this new documentation is the base directory for relative paths while the rest is context to understand it and the limitations (maybe I'm being biased by my workflows).

@tmandry
Copy link
Member

tmandry commented Dec 18, 2024

@rfcbot reviewed

@traviscross
Copy link
Contributor

@rustbot labels -I-lang-nominated

We discussed this in our lang call today. It's now in FCP.

@rustbot rustbot removed the I-lang-nominated Nominated for discussion during a lang team meeting. label Dec 19, 2024
/// The file name is derived from the root module's source path passed to the Rust compiler
/// (e.g. `./src/lib.rs`)
/// end the sequence the compiler takes to get from root module to the module containing `file!`.
/// (e.g. inside `mod foo;` in `./src/lib.rs`, `file!` would be `./src/foo.rs`),
Copy link
Contributor

@traviscross traviscross Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...or ./src/foo/mod.rs :) Perhaps something like:

Suggested change
/// (e.g. inside `mod foo;` in `./src/lib.rs`, `file!` would be `./src/foo.rs`),
/// (e.g. if the root file of the crate is `./src/lib.rs`, and within that
/// crate there is a module `foo` whose source file is `./src/foo.rs`, then
/// `file!` called within will return `./src/foo.rs`),

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since there is a complete sentence before this, I think the parentheses could be removed and "e.g." replaced with "For example,".

As-is this parenthetical clause starts a sentence and then is followed by , modified by any..., I think that trailer is meant to go with the first sentence (before this example) in either case.

Copy link
Contributor

@traviscross traviscross Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The full stop ahead of the parenthetical seems an error, as the sentence does continue after the parenthetical. As you say, probably it'd be better to keep the sentence together and state the "For example" after it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled the example out into a sentence at the end so that the longer example doesn't break the flow of the explanation.

@epage epage force-pushed the change branch 2 times, most recently from ed56c74 to 2d3a81a Compare December 19, 2024 19:51
///
/// The expanded expression has type `&'static str`, and the returned file is not the invocation
/// of the `file!` macro itself, but rather the first macro invocation leading up to the
/// invocation of the `file!` macro.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am just a daily rust user, not a compiler expert but I have used file macro before and I think this sentence is relatively hard to grasp. I think an example could help.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understood it as follows: If file!() is called from another macro, then it returns the name of the file where the outer macro is called, not the file where the literal file!() invocation is.

If that's correct, the original description is clear to me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if it looked like I was implying the documentation was wrong. That was not my intention, I was just asking for an example because it usually helps to understand things. If y'all think it's clear enough, feel free to resolve that thread.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it can be useful to find ways to improve documentation, this is out of scope of this PR. This was only called out because I didn't pay enough attention when I re-wrapped the lines and my editor and the author of this section's disagreed on the line wraps. I have since reverted that change.

@kornelski
Copy link
Contributor

The current implementation of location-detail=none may change the file path to be <redacted>. This probably should be documented too. Maybe the dummy value could be changed to something else.

There is a conflict between use of file! to discover or read paths from disk, and use of file! to embed error information: disk access and uses during tests/debugging want real accessible paths, but embedding of paths in released artefacts wants them simplified, anonymized, or completely removed.

@traviscross
Copy link
Contributor

traviscross commented Dec 27, 2024

@rfcbot concern tension-between-uses
@rustbot labels -I-lang-easy-decision +I-lang-nominated

There is a conflict between use of file! to discover or read paths from disk, and use of file! to embed error information...

That's a good point. Let's discuss that and confirm whether we're still happy here.

@rfcbot rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. and removed final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. labels Dec 27, 2024
@rustbot rustbot added I-lang-nominated Nominated for discussion during a lang team meeting. and removed I-lang-easy-decision Issue: The decision needed by the team is conjectured to be easy; this does not imply nomination labels Dec 27, 2024
@joshtriplett
Copy link
Member

I don't think those two uses conflict here. If you want to use a library (e.g. a testing framework) that relies on having paths in file! that provide enough information to find the original file, you shouldn't pass options that completely hide the information. (You can still use options that strip out parts of it, as long as your testing framework has enough to work with, and I think the defaults are likely to preserve enough information.)

The documentation says what the default behavior is, and notes that compiler options might change that behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. I-lang-nominated Nominated for discussion during a lang team meeting. proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-lang Relevant to the language team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.