Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle large chain calc better #17291

Merged
merged 4 commits into from
Jun 19, 2024
Merged

Handle large chain calc better #17291

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5a287d9
Handle large chain calc better
erikjohnston Jun 10, 2024
db67cd6
Newsfile
erikjohnston Jun 10, 2024
f237303
Handle previously persisted events properly
erikjohnston Jun 10, 2024
ef47167
Comments
erikjohnston Jun 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Handle previously persisted events properly
  • Loading branch information
@erikjohnston
erikjohnston committed Jun 10, 2024
commit f237303c32f866820df12d87d4d5579cc72cf7c6
25 changes: 24 additions & 1 deletion synapse/storage/databases/main/events.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,8 +254,15 @@ async def _persist_events_and_state_updates(
async def calculate_chain_cover_index_for_events(
self, room_id: str, events: Collection[EventBase]
) -> Dict[str, NewEventChainLinks]:
# Filter to state events, and ensure there are no duplicates.
state_events = []
seen_events = set()
for event in events:
if not event.is_state() or event.event_id in seen_events:
continue

state_events = [event for event in events if event.is_state()]
state_events.append(event)
seen_events.add(event.event_id)

if not state_events:
return {}
Expand Down Expand Up @@ -289,6 +296,22 @@ def calculate_chain_cover_index_for_events_txn(
if row is None:
return {}

# Filter out already persisted events.
rows = self.db_pool.simple_select_many_txn(
txn,
table="events",
column="event_id",
iterable=[e.event_id for e in state_events],
keyvalues={},
retcols=("event_id",),
)
already_persisted_events = {event_id for event_id, in rows}
state_events = [
event
for event in state_events
if event.event_id in already_persisted_events
]

if not state_events:
return {}

Expand Down
Loading