-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deduplicate references when some of them are in macro expansions #16358
Deduplicate references when some of them are in macro expansions #16358
Conversation
crates/ide/src/references.rs
Outdated
Some( | ||
find_defs(sema, &syntax, position.offset)? | ||
.into_iter() | ||
.unique() | ||
.map(search) | ||
.collect(), | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, this won't fix the linked issue though. In that issue, all the occurences are different Definition
s! We already do deduplicate the results, its just the the resulting NavigationTarget
s differ in properties that are not relevant to the shown results, so we need to do another deduplication pass when we convert to the requests response.
The deduplication should happen on the references before we return the ReferenceSearchResult
here:
rust-analyzer/crates/ide/src/references.rs
Lines 106 to 107 in 5df53c9
}); | |
ReferenceSearchResult { declaration, references } |
That is we need to go through all value Vec
s of the map, sort, and deduplicate them there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, this won't fix the linked issue though.
Good catch - that's because in that case the definitions are actually different, so unique() doesn't eliminate them, and because of the macro weirdness producing a cartesian product of references we also get redundant references along the technically-unique definitions.
The deduplication should happen on the references before we return the ReferenceSearchResult here:
The results of the lambda created by make_searcher
are already deduplicated. The problem is that we call said lambda multiple times and concatenate the results.
crates/ide/src/references.rs
Outdated
} | ||
fn test() { | ||
let name = (); | ||
use_name!(name$0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apparently our descend_into_macros() stanza returns one definition for
each time a name is used in a macro.
Sorry this was misleading. It's not about definitions but all references.
Given
macro_rules! use_name {
($name:ident) => {
println!("{}", $name);
println!("{}", $name);
println!("{}", $name);
};
}
fn test() {
let name = 123;
use_name!(name);
println!("{}", name);
println!("{}", name);
}
so 3 references inside the macro, and 4 outside;
this will produce 12 references; it's the product. Not sure if this is expected but I guess I haven't seen another problem with it.
It only surfaced when there was a definition inside a macro because we already deduplicate the per-definition references.
I think your suggestion makes sense; I'll try it and maybe get rid of the other two unique()
calls.
0fa227a
to
02d551c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah okay, now I see the general problem. In that case, find_all_refs
is working correctly. We already deduplicate references per Definition
, and these definitions have more semantic information so we don't wanna deduplicate any further at this stage. What we do wanna do is deduplicate at the LSP layer, as that's where most of this info becomes useless.
That is we want to deduplicate here https://github.com/veykril/rust-analyzer/blob/4cab41156ac820c83e7eb3488efd2be43f9fc7a7/crates/rust-analyzer/src/handlers/request.rs#L1067-L1090 and here https://github.com/veykril/rust-analyzer/blob/4cab41156ac820c83e7eb3488efd2be43f9fc7a7/crates/rust-analyzer/src/handlers/request.rs#L1807-L1815
02d551c
to
131b649
Compare
ok that makes sense |
fn deduplicate_declarations(refs: &mut Vec<ReferenceSearchResult>) { | ||
if refs.iter().filter(|decl| decl.declaration.is_some()).take(2).count() > 1 { | ||
let mut seen_declarations = HashSet::new(); | ||
refs.retain(|res| { | ||
res.declaration.as_ref().is_none_or(|decl| seen_declarations.insert(decl.nav.clone())) | ||
}); | ||
} | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may still end up with multiple overlapping links, we need to do the filtering after we turned the ReferenceSearchResult
s into Location
s, then dedup on those locations (this transformation, to_proto::location
is where we lose the unnecessary data that prevented the deduplication from working as intended in the first place).
That's why I linked to specify lines in my earlier review, sorry that I wasn't clear enough with that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should probably find a test case. My variable name seen_declarations
is misleading because we actually insert navigation targets here, which should be what we want to duplicate on?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No we do not want to deduplicate NavigationTarget
s, we want to deduplicate lsp Location
s. The nav targets haave a lot of info that differs even though the pointed to ranges are the same hence why deduplicating them is non trivial (and also wrong!). once we transformed them to locations we lose all the differing info of the nav targets that are irrelevant for the feature so now the deduplication does meaningful work (and is trivial to implement).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably do the same for goto_definition_response
131b649
to
27ed99b
Compare
27ed99b
to
daaa791
Compare
Commit 6a06f6f (Deduplicate reference search results, 2022-11-07) deduplicates references within each definition. There is an edge case when requesting references of a macro argument. Apparently, our descend_into_macros() stanza in references.rs produces a cartesian product of - references inside the macro times - times references outside the macro. Since the above deduplication only applies to the references within a single definition, we return them all, leading to many redundant references. Work around this by deduplicating definitions as well. Perhaps there is a better fix to not produce this cartesian product in the first place; but I think at least for definitions the problem would remain; a macro can contain multiple definitions of the same name, but since the navigation target will be the unresolved location, it's the same for all of them. We can't use unique() because we don't want to drop references that don't have a declaration (though I dont' have an example for this case). I discovered this working with the "bitflags" macro from the crate of the same name. Fixes rust-lang#16357
daaa791
to
91a8f34
Compare
Thanks! |
☀️ Test successful - checks-actions |
Deduplicating unconditionally seems not very elegant to me, that's why I prefer for a targeted fix. |
I personally find this approach a lot cleaner (and more importantly simpler!). We want to prevent to send duplicate entries in the LSP response from appearing, so it only makes sense to deduplicate at that layer, not prior to the lossy transformation. Deduplicating before we discard the unimportant bits means we might not deduplicate something that ends up the same post-lsp transformation. This doesn't mean it is necessarily the case right now (though I am fairly sure that can be achieved), but if
No worries, I am currently aggressively going through the open PRs to lower the number, otherwise I would've left this open until you had time to get back to this. |
EDIT: I wonder if this is a regression, I'll try to investigate.
Commit 6a06f6f (Deduplicate reference search results, 2022-11-07)
deduplicates references within each definition.
Apparently our descend_into_macros() stanza returns
one definition for each time a name is used in a macro.
Each of those definitions has the same set of references.
We return them all, leading to many redundant references.
Work around this by deduplicating definitions as well. Perhaps there
is a better fix to not produce duplicate definitions in the first
place.
I discovered this working with the "bitflags" macro from the crate
of the same name.
Fixes #16357