Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Combined modlog #5253

Merged
merged 101 commits into from
Jan 14, 2025
Merged

Adding Combined modlog #5253

merged 101 commits into from
Jan 14, 2025

Conversation

dessalines
Copy link
Member

@dessalines dessalines commented Dec 10, 2024

Context: #2444

Must come after #5251

Notes:

  • This removes all the individual modlog fetches into a single combined one, with a type_ filter.
  • 17 tables joined in total, with sometimes complicated joins required to fetch the derived data.
    • For example, mod_remove_post doesn't have the modded_person_id directly on it, it needs to get it from the post.creator_id column. I've tried to keep the joins as organized and well-commented as possible.
  • I've added a ton of tests, filtering different slices of all the filter types: community, post, comment, mod/admin, modded_person, and type. Its verbose, but its something we didn't have before.
  • Some of the mod views had missing fields that seemed important (occasionally it was missing a community or the modded person). I've added these.
  • I discovered a few errors and inconsistencies that I cleaned up.
  • There are plenty of breaking changes here, which includes the types coming back for the v3 routes. I am not going to do backwards compatibility for these types. That will have to be handled by API libraries switching on v.0.19 vs v.0.20.

dessalines and others added 30 commits November 26, 2024 09:27
* add pagination cursor

* store timestamp instead of id in cursor (partial)

* Revert "store timestamp instead of id in cursor (partial)"

This reverts commit 89359dd.

* use paginated query builder
- Separating the profile fetch from its combined content fetch.
- Starting to separate saved_only into its own combined view.
@dessalines
Copy link
Member Author

This is ready for review now.

cc @Nutomic @dullbananas

@dessalines dessalines marked this pull request as draft January 8, 2025 21:35
@dessalines dessalines marked this pull request as ready for review January 9, 2025 02:24
Comment on lines 1346 to 1352
let posts_mapped = &post_modlog.iter().filter_map(|p| {
if let ModlogCombinedView::ModRemovePost(v) = p {
Some(v)
} else {
None
}
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let posts_mapped = &post_modlog.iter().filter_map(|p| {
if let ModlogCombinedView::ModRemovePost(v) = p {
Some(v)
} else {
None
}
});
let posts_mapped = &post_modlog.iter().filter_map(|p| {
if let ModlogCombinedView::ModRemovePost(v) = p {
Some((v.mod_remove_post.removed, v.post.removed))
} else {
None
}
}).collect::<Vec<bool>>();

then replace the things below with a single assert_eq

same can be done for the other uses of filter_map

Copy link
Member Author

@dessalines dessalines Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

K done now. I didn't like doing it because its less clear now what they're doing, and these are just tests, where clarity should be preferred over code simplification.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I came up with a style that's more simple and clear than the other two:

assert!(matches!(
  &post_modlog[..],
  [
    ModlogCombinedView::ModRemovePost(ModRemovePostView {
      mod_remove_post: ModRemovePost { removed: true, .. },
      post: Post { removed: true, .. },
      ..
    }),
    ModlogCombinedView::ModRemovePost(ModRemovePostView {
      mod_remove_post: ModRemovePost { removed: true, .. },
      post: Post { removed: true, .. },
      ..
    }),
  ],
));

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That does look much cleaner, I'll try it out.

"ModRemoveComment" => query.filter(modlog_combined::mod_remove_comment_id.eq(id)),
"ModRemoveCommunity" => query.filter(modlog_combined::mod_remove_community_id.eq(id)),
"ModRemovePost" => query.filter(modlog_combined::mod_remove_post_id.eq(id)),
"ModTransferCommunity" => query.filter(modlog_combined::mod_transfer_community_id.eq(id)),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EnumDiscriminants might help

crates/db_views_moderator/src/modlog_combined_view.rs Outdated Show resolved Hide resolved
crates/db_views_moderator/src/modlog_combined_view.rs Outdated Show resolved Hide resolved
crates/db_views_moderator/src/modlog_combined_view.rs Outdated Show resolved Hide resolved
@dessalines
Copy link
Member Author

dessalines commented Jan 10, 2025

Probably not worth it to do any of the enum work. All the other combined cases just use a single char for the paging, this one's unique in that there are 17 types it needs to combine.

EDIT: I'm reverting adding strum at all because of this.

@Nutomic Nutomic merged commit 9c1347c into main Jan 14, 2025
2 checks passed
@SleeplessOne1917 SleeplessOne1917 deleted the combined_modlog branch January 15, 2025 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants