MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC #3861

hughns · 2022-08-04T16:50:31Z

Rendered

Dependencies/sub-proposals:

proposals/3861-delegated-oidc-architecture.md

MTRNord · 2022-08-09T14:07:25Z

proposals/3861-delegated-oidc-architecture.md

+1. Accept the set of MSCs to enable delegation via OIDC.
+1. Deprecate non-OIDC auth related API endpoints or capabilities in existing Matrix APIs.
+1. Provide migration support to the ecosystem.
+1. Close all existing MSCs relating to non-OIDC as `obsolete`.


Please make sure to check if those are actually solved by this when doing it. There might be some around that OIDC does not fix but actually complicates, like the Concealed login MSC.

That is a good point. We should spend some time identifying those MSCs sooner than later, and see for each MSC if either:

it solves the problem solved by the MSC

the tradeoffs are acceptable if it is conceptually orthogonal

Since now half a year went by: is there any update on which mscs are compatible and which aren't?

@MTRNord there is a start of a list at https://github.com/matrix-org/matrix-spec-proposals/blob/hughns/delegated-oidc-architecture/proposals/3861-delegated-oidc-architecture.md#alternatives

…ndation

turt2live

just a mid-read review. Submitting before I forget about it

proposals/3861-delegated-oidc-architecture.md

Co-authored-by: greizgh <greizgh@ephax.org>

MTRNord · 2023-02-24T16:21:57Z

proposals/3861-delegated-oidc-architecture.md

+
+## Security considerations
+
+Please refer to individual proposals.


I am not sure if there is a better place, but:

How do we protect from ending up in the same situation as email, where big companies (like google/microsoft) end up using OIDC/OAuth2 purely to gatekeep the clients that are allowed to use? As far as I understand, matrix is supposed to be an open protocol where anyone is allowed to bring their own tools and software. So how do we make sure that this is still possible? I am aware of certain enterprise parts of matrix that want to restrict client usage, however imho this isn't something that should leak into spec.

The reasoning behind is that the Matrix Manifesto ( https://matrix.org/foundation/ ) clearly defines that we do want to be an open communication network and people shall have full control over their communication. If we end up however in a situation where we give HS admins a (too) easy way to restrict the clients that are usable we would break the goals of that manifesto at it's core, since then it's not open anymore for the users and Singular community managers can decide what tools to use with 0 protection for the users.

(edit: typos)

For additional source of my email reference, see the comment made by a thunderbird dev in https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat#OAuth2

Due to a defficiency in the OAuth2 spec, the client is usually required to send a client credential key, which in turn requires the client to be registered and approved by the email provider. Unfortunately, this not only allows email providers to block specific email clients (which is contrary to the idea of Open-Source), but also makes it impossible to support arbitrary OAuth2 servers. That's why Thunderbird is forced to hardcode the servers that it supports and the respecive client keys. That means that you cannot use OAuth2 for your own server. Only the servers listed on OAuth2Providers.jsm will work.

I am aware of the dynamic registration MSC for OIDC, but that only somewhat solves this, since there a provider still is able to refuse your client at any given time without explanation.

I'm not sure how to properly address this to be honest. MSC2966 outlines what the client must advertise, and what the homeserver should accept. If the HS doesn't accept a valid registration, I would argue that this HS is no longer Matrix-compliant on the C-S API side?

This was always a possibility in the ecosystem: nothing prevented a homeserver to have an allow-list of valid redirect URLs for the m.login.sso flow.
What this proposal change, is that the homeserver has more metadata on the client than before to work with, which is a good thing for giving homeserver admins and users a better idea of what is happening with their account, who has access to it.

Note that the notes from Thunderbird are completely valid for email, but not for Matrix, as this 'many client, many homeservers' constraint means we have to have some sort of open client registration. Mastodon has something similar, Bluesky/ATProto as well… I'm not sure why it would not work for Matrix

I think it'd be fine to acknowledge the concern in the MSC, but I don't forsee a way to fix it. Homeservers can (and do) already gatekeep their registration and login APIs to particular clients/domains for a variety of reasons, including user safety and security - they might relax these conditions if given a proper authentication interface to use.

hughns · 2023-02-28T16:54:37Z

I've added references to the following MSCs that this proposal assumes are generally accepted:

clokep · 2023-05-26T15:03:51Z

proposals/3861-delegated-oidc-architecture.md

+| [MSC2967](https://github.com/matrix-org/matrix-doc/pull/2967) | Defines the namespace for a set of API scopes that can can expanded in future to allow for finer grained permissioning |
+| [MSC3824](https://github.com/matrix-org/matrix-doc/pull/3824) | Proposes some minor changes to the C-S API to allow Matrix clients that are not fully OIDC-native to work best with an OIDC enabled homeserver that has is serving a compatibility layer |
+
+## Potential issues


From reviewing the Synapse implementation it seems that this MSC is missing information on how application services would register users. matrix-org/synapse#15582 keeps the old /register endpoint active for the m.login.application_service login type only.

This MSC needs to include some information IMO about how this interacts with application services.

Synapse 1.86.0 (2023-06-20) =========================== No significant changes since 1.86.0rc2. Synapse 1.86.0rc2 (2023-06-14) ============================== Bugfixes -------- - Fix an error when having workers of different versions running. ([\matrix-org#15774](matrix-org#15774)) Synapse 1.86.0rc1 (2023-06-13) ============================== This version was tagged but never released. Features -------- - Stable support for [MSC3882](matrix-org/matrix-spec-proposals#3882) to allow an existing device/session to generate a login token for use on a new device/session. ([\matrix-org#15388](matrix-org#15388)) - Support resolving a room's [canonical alias](https://spec.matrix.org/v1.7/client-server-api/#mroomcanonical_alias) via the module API. ([\matrix-org#15450](matrix-org#15450)) - Enable support for [MSC3952](matrix-org/matrix-spec-proposals#3952): intentional mentions. ([\matrix-org#15520](matrix-org#15520)) - Experimental [MSC3861](matrix-org/matrix-spec-proposals#3861) support: delegate auth to an OIDC provider. ([\matrix-org#15582](matrix-org#15582)) - Add Synapse version deploy annotations to Grafana dashboard which enables easy correlation between behavior changes witnessed in a graph to a certain Synapse version and nail down regressions. ([\matrix-org#15674](matrix-org#15674)) - Add a catch-all * to the supported relation types when redacting an event and its related events. This is an update to [MSC3912](matrix-org/matrix-spec-proposals#3861) implementation. ([\matrix-org#15705](matrix-org#15705)) - Speed up `/messages` by backfilling in the background when there are no backward extremities where we are directly paginating. ([\matrix-org#15710](matrix-org#15710)) - Expose a metric reporting the database background update status. ([\matrix-org#15740](matrix-org#15740)) Bugfixes -------- - Correctly clear caches when we delete a room. ([\matrix-org#15609](matrix-org#15609)) - Check permissions for enabling encryption earlier during room creation to avoid creating broken rooms. ([\matrix-org#15695](matrix-org#15695)) Improved Documentation ---------------------- - Simplify query to find participating servers in a room. ([\matrix-org#15732](matrix-org#15732)) Internal Changes ---------------- - Log when events are (maybe unexpectedly) filtered out of responses in tests. ([\matrix-org#14213](matrix-org#14213)) - Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters`. ([\matrix-org#15649](matrix-org#15649)) - Add support for tracing functions which return `Awaitable`s. ([\matrix-org#15650](matrix-org#15650)) - Cache requests for user's devices over federation. ([\matrix-org#15675](matrix-org#15675)) - Add fully qualified docker image names to Dockerfiles. ([\matrix-org#15689](matrix-org#15689)) - Remove some unused code. ([\matrix-org#15690](matrix-org#15690)) - Improve type hints. ([\matrix-org#15694](matrix-org#15694), [\matrix-org#15697](matrix-org#15697)) - Update docstring and traces on `maybe_backfill()` functions. ([\matrix-org#15709](matrix-org#15709)) - Add context for when/why to use the `long_retries` option when sending Federation requests. ([\matrix-org#15721](matrix-org#15721)) - Removed some unused fields. ([\matrix-org#15723](matrix-org#15723)) - Update federation error to more plainly explain we can only authorize our own membership events. ([\matrix-org#15725](matrix-org#15725)) - Prevent the `latest_deps` and `twisted_trunk` daily GitHub Actions workflows from running on forks of the codebase. ([\matrix-org#15726](matrix-org#15726)) - Improve performance of user directory search. ([\matrix-org#15729](matrix-org#15729)) - Remove redundant table join with `room_memberships` when doing a `is_host_joined()`/`is_host_invited()` call (`membership` is already part of the `current_state_events`). ([\matrix-org#15731](matrix-org#15731)) - Remove superfluous `room_memberships` join from background update. ([\matrix-org#15733](matrix-org#15733)) - Speed up typechecking CI. ([\matrix-org#15752](matrix-org#15752)) - Bump minimum supported Rust version to 1.60.0. ([\matrix-org#15768](matrix-org#15768)) * Bump importlib-metadata from 6.1.0 to 6.6.0. ([\matrix-org#15711](matrix-org#15711)) * Bump library/redis from 6-bullseye to 7-bullseye in /docker. ([\matrix-org#15712](matrix-org#15712)) * Bump log from 0.4.18 to 0.4.19. ([\matrix-org#15761](matrix-org#15761)) * Bump phonenumbers from 8.13.11 to 8.13.13. ([\matrix-org#15763](matrix-org#15763)) * Bump pyasn1 from 0.4.8 to 0.5.0. ([\matrix-org#15713](matrix-org#15713)) * Bump pydantic from 1.10.8 to 1.10.9. ([\matrix-org#15762](matrix-org#15762)) * Bump pyo3-log from 0.8.1 to 0.8.2. ([\matrix-org#15759](matrix-org#15759)) * Bump pyopenssl from 23.1.1 to 23.2.0. ([\matrix-org#15765](matrix-org#15765)) * Bump regex from 1.7.3 to 1.8.4. ([\matrix-org#15769](matrix-org#15769)) * Bump sentry-sdk from 1.22.1 to 1.25.0. ([\matrix-org#15714](matrix-org#15714)) * Bump sentry-sdk from 1.25.0 to 1.25.1. ([\matrix-org#15764](matrix-org#15764)) * Bump serde from 1.0.163 to 1.0.164. ([\matrix-org#15760](matrix-org#15760)) * Bump types-jsonschema from 4.17.0.7 to 4.17.0.8. ([\matrix-org#15716](matrix-org#15716)) * Bump types-pyopenssl from 23.1.0.2 to 23.2.0.0. ([\matrix-org#15766](matrix-org#15766)) * Bump types-requests from 2.31.0.0 to 2.31.0.1. ([\matrix-org#15715](matrix-org#15715))

@Fizzadar

Synapse 1.86.0 (2023-06-20) =========================== No significant changes since 1.86.0rc2. Synapse 1.86.0rc2 (2023-06-14) ============================== Bugfixes -------- - Fix an error when having workers of different versions running. ([\#15774](matrix-org/synapse#15774)) Synapse 1.86.0rc1 (2023-06-13) ============================== This version was tagged but never released. Features -------- - Stable support for [MSC3882](matrix-org/matrix-spec-proposals#3882) to allow an existing device/session to generate a login token for use on a new device/session. ([\#15388](matrix-org/synapse#15388)) - Support resolving a room's [canonical alias](https://spec.matrix.org/v1.7/client-server-api/#mroomcanonical_alias) via the module API. ([\#15450](matrix-org/synapse#15450)) - Enable support for [MSC3952](matrix-org/matrix-spec-proposals#3952): intentional mentions. ([\#15520](matrix-org/synapse#15520)) - Experimental [MSC3861](matrix-org/matrix-spec-proposals#3861) support: delegate auth to an OIDC provider. ([\#15582](matrix-org/synapse#15582)) - Add Synapse version deploy annotations to Grafana dashboard which enables easy correlation between behavior changes witnessed in a graph to a certain Synapse version and nail down regressions. ([\#15674](matrix-org/synapse#15674)) - Add a catch-all * to the supported relation types when redacting an event and its related events. This is an update to [MSC3912](matrix-org/matrix-spec-proposals#3861) implementation. ([\#15705](matrix-org/synapse#15705)) - Speed up `/messages` by backfilling in the background when there are no backward extremities where we are directly paginating. ([\#15710](matrix-org/synapse#15710)) - Expose a metric reporting the database background update status. ([\#15740](matrix-org/synapse#15740)) Bugfixes -------- - Correctly clear caches when we delete a room. ([\#15609](matrix-org/synapse#15609)) - Check permissions for enabling encryption earlier during room creation to avoid creating broken rooms. ([\#15695](matrix-org/synapse#15695)) Improved Documentation ---------------------- - Simplify query to find participating servers in a room. ([\#15732](matrix-org/synapse#15732)) Internal Changes ---------------- - Log when events are (maybe unexpectedly) filtered out of responses in tests. ([\#14213](matrix-org/synapse#14213)) - Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters`. ([\#15649](matrix-org/synapse#15649)) - Add support for tracing functions which return `Awaitable`s. ([\#15650](matrix-org/synapse#15650)) - Cache requests for user's devices over federation. ([\#15675](matrix-org/synapse#15675)) - Add fully qualified docker image names to Dockerfiles. ([\#15689](matrix-org/synapse#15689)) - Remove some unused code. ([\#15690](matrix-org/synapse#15690)) - Improve type hints. ([\#15694](matrix-org/synapse#15694), [\#15697](matrix-org/synapse#15697)) - Update docstring and traces on `maybe_backfill()` functions. ([\#15709](matrix-org/synapse#15709)) - Add context for when/why to use the `long_retries` option when sending Federation requests. ([\#15721](matrix-org/synapse#15721)) - Removed some unused fields. ([\#15723](matrix-org/synapse#15723)) - Update federation error to more plainly explain we can only authorize our own membership events. ([\#15725](matrix-org/synapse#15725)) - Prevent the `latest_deps` and `twisted_trunk` daily GitHub Actions workflows from running on forks of the codebase. ([\#15726](matrix-org/synapse#15726)) - Improve performance of user directory search. ([\#15729](matrix-org/synapse#15729)) - Remove redundant table join with `room_memberships` when doing a `is_host_joined()`/`is_host_invited()` call (`membership` is already part of the `current_state_events`). ([\#15731](matrix-org/synapse#15731)) - Remove superfluous `room_memberships` join from background update. ([\#15733](matrix-org/synapse#15733)) - Speed up typechecking CI. ([\#15752](matrix-org/synapse#15752)) - Bump minimum supported Rust version to 1.60.0. ([\#15768](matrix-org/synapse#15768)) ### Updates to locked dependencies * Bump importlib-metadata from 6.1.0 to 6.6.0. ([\#15711](matrix-org/synapse#15711)) * Bump library/redis from 6-bullseye to 7-bullseye in /docker. ([\#15712](matrix-org/synapse#15712)) * Bump log from 0.4.18 to 0.4.19. ([\#15761](matrix-org/synapse#15761)) * Bump phonenumbers from 8.13.11 to 8.13.13. ([\#15763](matrix-org/synapse#15763)) * Bump pyasn1 from 0.4.8 to 0.5.0. ([\#15713](matrix-org/synapse#15713)) * Bump pydantic from 1.10.8 to 1.10.9. ([\#15762](matrix-org/synapse#15762)) * Bump pyo3-log from 0.8.1 to 0.8.2. ([\#15759](matrix-org/synapse#15759)) * Bump pyopenssl from 23.1.1 to 23.2.0. ([\#15765](matrix-org/synapse#15765)) * Bump regex from 1.7.3 to 1.8.4. ([\#15769](matrix-org/synapse#15769)) * Bump sentry-sdk from 1.22.1 to 1.25.0. ([\#15714](matrix-org/synapse#15714)) * Bump sentry-sdk from 1.25.0 to 1.25.1. ([\#15764](matrix-org/synapse#15764)) * Bump serde from 1.0.163 to 1.0.164. ([\#15760](matrix-org/synapse#15760)) * Bump types-jsonschema from 4.17.0.7 to 4.17.0.8. ([\#15716](matrix-org/synapse#15716)) * Bump types-pyopenssl from 23.1.0.2 to 23.2.0.0. ([\#15766](matrix-org/synapse#15766)) * Bump types-requests from 2.31.0.0 to 2.31.0.1. ([\#15715](matrix-org/synapse#15715)) Synapse 1.85.2 (2023-06-08) =========================== Bugfixes -------- - Fix regression where using TLS for HTTP replication between workers did not work. Introduced in v1.85.0. ([\#15746](matrix-org/synapse#15746)) Synapse 1.85.1 (2023-06-07) =========================== Note: this release only fixes a bug that stopped some deployments from upgrading to v1.85.0. There is no need to upgrade to v1.85.1 if successfully running v1.85.0. Bugfixes -------- - Fix bug in schema delta that broke upgrades for some deployments. Introduced in v1.85.0. ([\#15738](matrix-org/synapse#15738), [\#15739](matrix-org/synapse#15739)) Synapse 1.85.0 (2023-06-06) =========================== No significant changes since 1.85.0rc2. ## Security advisory The following issues are fixed in 1.85.0 (and RCs). - [GHSA-26c5-ppr8-f33p](GHSA-26c5-ppr8-f33p) / [CVE-2023-32682](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-32682) — Low Severity It may be possible for a deactivated user to login when using uncommon configurations. - [GHSA-98px-6486-j7qc](GHSA-98px-6486-j7qc) / [CVE-2023-32683](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-32683) — Low Severity A discovered oEmbed or image URL can bypass the `url_preview_url_blacklist` setting potentially allowing server side request forgery or bypassing network policies. Impact is limited to IP addresses allowed by the `url_preview_ip_range_blacklist` setting (by default this only allows public IPs). See the advisories for more details. If you have any questions, email security@matrix.org. Synapse 1.85.0rc2 (2023-06-01) ============================== Bugfixes -------- - Fix a performance issue introduced in Synapse v1.83.0 which meant that purging rooms was very slow and database-intensive. ([\#15693](matrix-org/synapse#15693)) Deprecations and Removals ------------------------- - Deprecate calling the `/register` endpoint with an unspecced `user` property for application services. ([\#15703](matrix-org/synapse#15703)) Internal Changes ---------------- - Speed up background jobs `populate_full_user_id_user_filters` and `populate_full_user_id_profiles`. ([\#15700](matrix-org/synapse#15700)) Synapse 1.85.0rc1 (2023-05-30) ============================== Features -------- - Improve performance of backfill requests by performing backfill of previously failed requests in the background. ([\#15585](matrix-org/synapse#15585)) - Add a new [admin API](https://matrix-org.github.io/synapse/v1.85/usage/administration/admin_api/index.html) to [create a new device for a user](https://matrix-org.github.io/synapse/v1.85/admin_api/user_admin_api.html#create-a-device). ([\#15611](matrix-org/synapse#15611)) - Add Unix socket support for Redis connections. Contributed by Jason Little. ([\#15644](matrix-org/synapse#15644)) Bugfixes -------- - Fix a long-standing bug where setting the read marker could fail when using message retention. Contributed by Nick @ Beeper (@Fizzadar). ([\#15464](matrix-org/synapse#15464)) - Fix a long-standing bug where the `url_preview_url_blacklist` configuration setting was not applied to oEmbed or image URLs found while previewing a URL. ([\#15601](matrix-org/synapse#15601)) - Fix a long-standing bug where filters with multiple backslashes were rejected. ([\#15607](matrix-org/synapse#15607)) - Fix a bug introduced in Synapse 1.82.0 where the error message displayed when validation of the `app_service_config_files` config option fails would be incorrectly formatted. ([\#15614](matrix-org/synapse#15614)) - Fix a long-standing bug where deactivated users were still able to login using the custom `org.matrix.login.jwt` login type (if enabled). ([\#15624](matrix-org/synapse#15624)) - Fix a long-standing bug where deactivated users were able to login in uncommon situations. ([\#15634](matrix-org/synapse#15634)) Improved Documentation ---------------------- - Warn users that at least 3.75GB of space is needed for the nix Synapse development environment. ([\#15613](matrix-org/synapse#15613)) - Remove outdated comment from the generated and sample homeserver log configs. ([\#15648](matrix-org/synapse#15648)) - Improve contributor docs to make it more clear that Rust is a necessary prerequisite. Contributed by @grantm. ([\#15668](matrix-org/synapse#15668)) Deprecations and Removals ------------------------- - Remove the old version of the R30 (30-day retained users) phone-home metric. ([\#10428](matrix-org/synapse#10428)) Internal Changes ---------------- - Create dependabot changelogs at release time. ([\#15481](matrix-org/synapse#15481)) - Add not null constraint to column `full_user_id` of tables `profiles` and `user_filters`. ([\#15537](matrix-org/synapse#15537)) - Allow connecting to HTTP Replication Endpoints by using `worker_name` when constructing the request. ([\#15578](matrix-org/synapse#15578)) - Make the `thread_id` column on `event_push_actions`, `event_push_actions_staging`, and `event_push_summary` non-null. ([\#15597](matrix-org/synapse#15597)) - Run mypy type checking with the minimum supported Python version to catch new usage that isn't backwards-compatible. ([\#15602](matrix-org/synapse#15602)) - Fix subscriptable type usage in Python <3.9. ([\#15604](matrix-org/synapse#15604)) - Update internal terminology. ([\#15606](matrix-org/synapse#15606), [\#15620](matrix-org/synapse#15620)) - Instrument `state` and `state_group` storage-related operations to better picture what's happening when tracing. ([\#15610](matrix-org/synapse#15610), [\#15647](matrix-org/synapse#15647)) - Trace how many new events from the backfill response we need to process. ([\#15633](matrix-org/synapse#15633)) - Re-type config paths in `ConfigError`s to be `StrSequence`s instead of `Iterable[str]`s. ([\#15615](matrix-org/synapse#15615)) - Update Mutual Rooms ([MSC2666](matrix-org/matrix-spec-proposals#2666)) implementation to match new proposal text. ([\#15621](matrix-org/synapse#15621)) - Remove the unstable identifiers from faster joins ([MSC3706](matrix-org/matrix-spec-proposals#3706)). ([\#15625](matrix-org/synapse#15625)) - Fix the olddeps CI. ([\#15626](matrix-org/synapse#15626)) - Remove duplicate timestamp from test logs (`_trial_temp/test.log`). ([\#15636](matrix-org/synapse#15636)) - Fix two memory leaks in `trial` test runs. ([\#15630](matrix-org/synapse#15630)) - Limit the size of the `HomeServerConfig` cache in trial test runs. ([\#15646](matrix-org/synapse#15646)) - Improve type hints. ([\#15658](matrix-org/synapse#15658), [\#15659](matrix-org/synapse#15659)) - Add requesting user id parameter to key claim methods in `TransportLayerClient`. ([\#15663](matrix-org/synapse#15663)) - Speed up rebuilding of the user directory for local users. ([\#15665](matrix-org/synapse#15665)) - Implement "option 2" for [MSC3820](matrix-org/matrix-spec-proposals#3820): Room version 11. ([\#15666](matrix-org/synapse#15666), [\#15678](matrix-org/synapse#15678)) ### Updates to locked dependencies * Bump furo from 2023.3.27 to 2023.5.20. ([\#15642](matrix-org/synapse#15642)) * Bump log from 0.4.17 to 0.4.18. ([\#15681](matrix-org/synapse#15681)) * Bump prometheus-client from 0.16.0 to 0.17.0. ([\#15682](matrix-org/synapse#15682)) * Bump pydantic from 1.10.7 to 1.10.8. ([\#15685](matrix-org/synapse#15685)) * Bump pygithub from 1.58.1 to 1.58.2. ([\#15643](matrix-org/synapse#15643)) * Bump requests from 2.28.2 to 2.31.0. ([\#15651](matrix-org/synapse#15651)) * Bump sphinx from 6.1.3 to 6.2.1. ([\#15641](matrix-org/synapse#15641)) * Bump types-bleach from 6.0.0.1 to 6.0.0.3. ([\#15686](matrix-org/synapse#15686)) * Bump types-pillow from 9.5.0.2 to 9.5.0.4. ([\#15640](matrix-org/synapse#15640)) * Bump types-pyyaml from 6.0.12.9 to 6.0.12.10. ([\#15683](matrix-org/synapse#15683)) * Bump types-requests from 2.30.0.0 to 2.31.0.0. ([\#15684](matrix-org/synapse#15684)) * Bump types-setuptools from 67.7.0.2 to 67.8.0.0. ([\#15639](matrix-org/synapse#15639)) Synapse 1.84.1 (2023-05-26) =========================== This patch release fixes a major issue with homeservers that do not have an `instance_map` defined but which do use workers. If you have already upgraded to Synapse 1.84.0 and your homeserver is working normally, then there is no need to update to this patch release. Bugfixes -------- - Fix a bug introduced in Synapse v1.84.0 where workers do not start up when no `instance_map` was provided. ([\#15672](matrix-org/synapse#15672)) Internal Changes ---------------- - Add `dch` and `notify-send` to the development Nix flake so that the release script can be used. ([\#15673](matrix-org/synapse#15673)) Synapse 1.84.0 (2023-05-23) =========================== The `worker_replication_*` configuration settings have been deprecated in favour of configuring the main process consistently with other instances in the `instance_map`. The deprecated settings will be removed in Synapse v1.88.0, but changing your configuration in advance is recommended. See the [upgrade notes](https://github.com/matrix-org/synapse/blob/release-v1.84/docs/upgrade.md#upgrading-to-v1840) for more information. Bugfixes -------- - Fix a bug introduced in Synapse 1.84.0rc1 where errors during startup were not reported correctly on Python < 3.10. ([\#15599](matrix-org/synapse#15599)) Synapse 1.84.0rc1 (2023-05-16) ============================== Features -------- - Add an option to prevent media downloads from configured domains. ([\#15197](matrix-org/synapse#15197)) - Add `forget_rooms_on_leave` config option to automatically forget rooms when users leave them or are removed from them. ([\#15224](matrix-org/synapse#15224)) - Add redis TLS configuration options. ([\#15312](matrix-org/synapse#15312)) - Add a config option to delay push notifications by a random amount, to discourage time-based profiling. ([\#15516](matrix-org/synapse#15516)) - Stabilize support for [MSC2659](matrix-org/matrix-spec-proposals#2659): application service ping endpoint. Contributed by Tulir @ Beeper. ([\#15528](matrix-org/synapse#15528)) - Implement [MSC4009](matrix-org/matrix-spec-proposals#4009) to expand the supported characters in Matrix IDs. ([\#15536](matrix-org/synapse#15536)) - Advertise support for Matrix 1.6 on `/_matrix/client/versions`. ([\#15559](matrix-org/synapse#15559)) - Print full error and stack-trace of any exception that occurs during startup/initialization. ([\#15569](matrix-org/synapse#15569)) Bugfixes -------- - Don't fail on federation over TOR where SRV queries are not supported. Contributed by Zdzichu. ([\#15523](matrix-org/synapse#15523)) - Experimental support for [MSC4010](matrix-org/matrix-spec-proposals#4010) which rejects setting the `"m.push_rules"` via account data. ([\#15554](matrix-org/synapse#15554), [\#15555](matrix-org/synapse#15555)) - Fix a long-standing bug where an invalid membership event could cause an internal server error. ([\#15564](matrix-org/synapse#15564)) - Require at least poetry-core v1.1.0. ([\#15566](matrix-org/synapse#15566), [\#15571](matrix-org/synapse#15571)) Deprecations and Removals ------------------------- - Remove need for `worker_replication_*` based settings in worker configuration yaml by placing this data directly on the `instance_map` instead. ([\#15491](matrix-org/synapse#15491)) Updates to the Docker image --------------------------- - Add pkg-config package to Stage 0 to be able to build Dockerfile on ppc64le architecture. ([\#15567](matrix-org/synapse#15567)) Improved Documentation ---------------------- - Clarify documentation of the "Create or modify account" Admin API. ([\#15544](matrix-org/synapse#15544)) - Fix path to the `statistics/database/rooms` admin API in documentation. ([\#15560](matrix-org/synapse#15560)) - Update and improve Mastodon Single Sign-On documentation. ([\#15587](matrix-org/synapse#15587)) Internal Changes ---------------- - Use oEmbed to generate URL previews for YouTube Shorts. ([\#15025](matrix-org/synapse#15025)) - Create new `Client` for use with HTTP Replication between workers. Contributed by Jason Little. ([\#15470](matrix-org/synapse#15470)) - Bump pyicu from 2.10.2 to 2.11. ([\#15509](matrix-org/synapse#15509)) - Remove references to supporting per-user flag for [MSC2654](matrix-org/matrix-spec-proposals#2654). ([\#15522](matrix-org/synapse#15522)) - Don't use a trusted key server when running the demo scripts. ([\#15527](matrix-org/synapse#15527)) - Speed up rebuilding of the user directory for local users. ([\#15529](matrix-org/synapse#15529)) - Speed up deleting of old rows in `event_push_actions`. ([\#15531](matrix-org/synapse#15531)) - Install the `xmlsec` and `mdbook` packages and switch back to the upstream [cachix/devenv](https://github.com/cachix/devenv) repo in the nix development environment. ([\#15532](matrix-org/synapse#15532), [\#15533](matrix-org/synapse#15533), [\#15545](matrix-org/synapse#15545)) - Implement [MSC3987](matrix-org/matrix-spec-proposals#3987) by removing `"dont_notify"` from the list of actions in default push rules. ([\#15534](matrix-org/synapse#15534)) - Move various module API callback registration methods to a dedicated class. ([\#15535](matrix-org/synapse#15535)) - Proxy `/user/devices` federation queries to application services for [MSC3984](matrix-org/matrix-spec-proposals#3984). ([\#15539](matrix-org/synapse#15539)) - Factor out an `is_mine_server_name` method. ([\#15542](matrix-org/synapse#15542)) - Allow running Complement tests using [podman](https://podman.io/) by adding a `PODMAN` environment variable to `scripts-dev/complement.sh`. ([\#15543](matrix-org/synapse#15543)) - Bump serde from 1.0.160 to 1.0.162. ([\#15548](matrix-org/synapse#15548)) - Bump types-setuptools from 67.6.0.5 to 67.7.0.1. ([\#15549](matrix-org/synapse#15549)) - Bump sentry-sdk from 1.19.1 to 1.22.1. ([\#15550](matrix-org/synapse#15550)) - Bump ruff from 0.0.259 to 0.0.265. ([\#15551](matrix-org/synapse#15551)) - Bump hiredis from 2.2.2 to 2.2.3. ([\#15552](matrix-org/synapse#15552)) - Bump types-requests from 2.29.0.0 to 2.30.0.0. ([\#15553](matrix-org/synapse#15553)) - Add `org.matrix.msc3981` info to `/_matrix/client/versions`. ([\#15558](matrix-org/synapse#15558)) - Declare unstable support for [MSC3391](matrix-org/matrix-spec-proposals#3391) under `/_matrix/client/versions` if the experimental implementation is enabled. ([\#15562](matrix-org/synapse#15562)) - Implement [MSC3821](matrix-org/matrix-spec-proposals#3821) to update the redaction rules. ([\#15563](matrix-org/synapse#15563)) - Implement updated redaction rules from [MSC3389](matrix-org/matrix-spec-proposals#3389). ([\#15565](matrix-org/synapse#15565)) - Allow `pip install` to use setuptools_rust 1.6.0 when building Synapse. ([\#15570](matrix-org/synapse#15570)) - Deal with upcoming Github Actions deprecations. ([\#15576](matrix-org/synapse#15576)) - Export `run_as_background_process` from the module API. ([\#15577](matrix-org/synapse#15577)) - Update build system requirements to allow building with poetry-core==1.6.0. ([\#15588](matrix-org/synapse#15588)) - Bump serde from 1.0.162 to 1.0.163. ([\#15589](matrix-org/synapse#15589)) - Bump phonenumbers from 8.13.7 to 8.13.11. ([\#15590](matrix-org/synapse#15590)) - Bump types-psycopg2 from 2.9.21.9 to 2.9.21.10. ([\#15591](matrix-org/synapse#15591)) - Bump types-commonmark from 0.9.2.2 to 0.9.2.3. ([\#15592](matrix-org/synapse#15592)) - Bump types-setuptools from 67.7.0.1 to 67.7.0.2. ([\#15594](matrix-org/synapse#15594)) Synapse 1.83.0 (2023-05-09) =========================== No significant changes since 1.83.0rc1. Synapse 1.83.0rc1 (2023-05-02) ============================== Features -------- - Experimental support to recursively provide relations per [MSC3981](matrix-org/matrix-spec-proposals#3981). ([\#15315](matrix-org/synapse#15315)) - Experimental support for [MSC3970](matrix-org/matrix-spec-proposals#3970): Scope transaction IDs to devices. ([\#15318](matrix-org/synapse#15318)) - Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/experimental_features.html) to support per-user feature flags. ([\#15344](matrix-org/synapse#15344)) - Add a module API to send an HTTP push notification. ([\#15387](matrix-org/synapse#15387)) - Add an [admin API endpoint](https://matrix-org.github.io/synapse/v1.83/admin_api/statistics.html#get-largest-rooms-by-size-in-database) to query the largest rooms by disk space used in the database. ([\#15482](matrix-org/synapse#15482)) Bugfixes -------- - Disable push rule evaluation for rooms excluded from sync. ([\#15361](matrix-org/synapse#15361)) - Fix a long-standing bug where cached server key results which were directly fetched would not be properly re-used. ([\#15417](matrix-org/synapse#15417)) - Fix a bug introduced in Synapse 1.73.0 where some experimental push rules were returned by default. ([\#15494](matrix-org/synapse#15494)) Improved Documentation ---------------------- - Add Nginx loadbalancing example with sticky mxid for workers. ([\#15411](matrix-org/synapse#15411)) - Update outdated development docs that mention restrictions in versions of SQLite that we no longer support. ([\#15498](matrix-org/synapse#15498)) Internal Changes ---------------- - Speedup tests by caching HomeServerConfig instances. ([\#15284](matrix-org/synapse#15284)) - Add denormalised event stream ordering column to membership state tables for future use. Contributed by Nick @ Beeper (@Fizzadar). ([\#15356](matrix-org/synapse#15356)) - Always use multi-user device resync replication endpoints. ([\#15418](matrix-org/synapse#15418)) - Add column `full_user_id` to tables `profiles` and `user_filters`. ([\#15458](matrix-org/synapse#15458)) - Update support for [MSC3983](matrix-org/matrix-spec-proposals#3983) to allow always returning fallback-keys in a `/keys/claim` request. ([\#15462](matrix-org/synapse#15462)) - Improve type hints. ([\#15465](matrix-org/synapse#15465), [\#15496](matrix-org/synapse#15496), [\#15497](matrix-org/synapse#15497)) - Support claiming more than one OTK at a time. ([\#15468](matrix-org/synapse#15468)) - Bump types-pyyaml from 6.0.12.8 to 6.0.12.9. ([\#15471](matrix-org/synapse#15471)) - Bump pyasn1-modules from 0.2.8 to 0.3.0. ([\#15473](matrix-org/synapse#15473)) - Bump cryptography from 40.0.1 to 40.0.2. ([\#15474](matrix-org/synapse#15474)) - Bump types-netaddr from 0.8.0.7 to 0.8.0.8. ([\#15475](matrix-org/synapse#15475)) - Bump types-jsonschema from 4.17.0.6 to 4.17.0.7. ([\#15476](matrix-org/synapse#15476)) - Ask bug reporters to provide logs as text. ([\#15479](matrix-org/synapse#15479)) - Add a Nix flake for use as a development environment. ([\#15495](matrix-org/synapse#15495)) - Bump anyhow from 1.0.70 to 1.0.71. ([\#15507](matrix-org/synapse#15507)) - Bump types-pillow from 9.4.0.19 to 9.5.0.2. ([\#15508](matrix-org/synapse#15508)) - Bump packaging from 23.0 to 23.1. ([\#15510](matrix-org/synapse#15510)) - Bump types-requests from 2.28.11.16 to 2.29.0.0. ([\#15511](matrix-org/synapse#15511)) - Bump setuptools-rust from 1.5.2 to 1.6.0. ([\#15512](matrix-org/synapse#15512)) - Update the check_schema_delta script to account for when the schema version has been bumped locally. ([\#15466](matrix-org/synapse#15466)) Synapse 1.82.0 (2023-04-25) =========================== No significant changes since 1.82.0rc1. Synapse 1.82.0rc1 (2023-04-18) ============================== Features -------- - Allow loading the `/directory/room/{roomAlias}` endpoint on workers. ([\#15333](matrix-org/synapse#15333)) - Add some validation to `instance_map` configuration loading. ([\#15431](matrix-org/synapse#15431)) - Allow loading the `/capabilities` endpoint on workers. ([\#15436](matrix-org/synapse#15436)) Bugfixes -------- - Delete server-side backup keys when deactivating an account. ([\#15181](matrix-org/synapse#15181)) - Fix and document untold assumption that `on_logged_out` module hooks will be called before the deletion of pushers. ([\#15410](matrix-org/synapse#15410)) - Improve robustness when handling a perspective key response by deduplicating received server keys. ([\#15423](matrix-org/synapse#15423)) - Synapse now correctly fails to start if the config option `app_service_config_files` is not a list. ([\#15425](matrix-org/synapse#15425)) - Disable loading `RefreshTokenServlet` (`/_matrix/client/(r0|v3|unstable)/refresh`) on workers. ([\#15428](matrix-org/synapse#15428)) Improved Documentation ---------------------- - Note that the `delete_stale_devices_after` background job always runs on the main process. ([\#15452](matrix-org/synapse#15452)) Deprecations and Removals ------------------------- - Remove the broken, unspecced registration fallback. Note that the *login* fallback is unaffected by this change. ([\#15405](matrix-org/synapse#15405)) Internal Changes ---------------- - Bump black from 23.1.0 to 23.3.0. ([\#15372](matrix-org/synapse#15372)) - Bump pyopenssl from 23.1.0 to 23.1.1. ([\#15373](matrix-org/synapse#15373)) - Bump types-psycopg2 from 2.9.21.8 to 2.9.21.9. ([\#15374](matrix-org/synapse#15374)) - Bump types-netaddr from 0.8.0.6 to 0.8.0.7. ([\#15375](matrix-org/synapse#15375)) - Bump types-opentracing from 2.4.10.3 to 2.4.10.4. ([\#15376](matrix-org/synapse#15376)) - Bump dawidd6/action-download-artifact from 2.26.0 to 2.26.1. ([\#15404](matrix-org/synapse#15404)) - Bump parameterized from 0.8.1 to 0.9.0. ([\#15412](matrix-org/synapse#15412)) - Bump types-pillow from 9.4.0.17 to 9.4.0.19. ([\#15413](matrix-org/synapse#15413)) - Bump sentry-sdk from 1.17.0 to 1.19.1. ([\#15414](matrix-org/synapse#15414)) - Bump immutabledict from 2.2.3 to 2.2.4. ([\#15415](matrix-org/synapse#15415)) - Bump dawidd6/action-download-artifact from 2.26.1 to 2.27.0. ([\#15441](matrix-org/synapse#15441)) - Bump serde_json from 1.0.95 to 1.0.96. ([\#15442](matrix-org/synapse#15442)) - Bump serde from 1.0.159 to 1.0.160. ([\#15443](matrix-org/synapse#15443)) - Bump pillow from 9.4.0 to 9.5.0. ([\#15444](matrix-org/synapse#15444)) - Bump furo from 2023.3.23 to 2023.3.27. ([\#15445](matrix-org/synapse#15445)) - Bump types-pyopenssl from 23.1.0.0 to 23.1.0.2. ([\#15446](matrix-org/synapse#15446)) - Bump mypy from 1.0.0 to 1.0.1. ([\#15447](matrix-org/synapse#15447)) - Bump psycopg2 from 2.9.5 to 2.9.6. ([\#15448](matrix-org/synapse#15448)) - Improve DB performance of clearing out old data from `stream_ordering_to_exterm`. ([\#15382](matrix-org/synapse#15382), [\#15429](matrix-org/synapse#15429)) - Implement [MSC3989](matrix-org/matrix-spec-proposals#3989) redaction algorithm. ([\#15393](matrix-org/synapse#15393)) - Implement [MSC2175](matrix-org/matrix-spec-proposals#2175) to stop adding `creator` to create events. ([\#15394](matrix-org/synapse#15394)) - Implement [MSC2174](matrix-org/matrix-spec-proposals#2174) to move the `redacts` key to a `content` property. ([\#15395](matrix-org/synapse#15395)) - Trust dtonlay/rust-toolchain in CI. ([\#15406](matrix-org/synapse#15406)) - Explicitly install Synapse during typechecking in CI. ([\#15409](matrix-org/synapse#15409)) - Only load the SSO redirect servlet if SSO is enabled. ([\#15421](matrix-org/synapse#15421)) - Refactor `SimpleHttpClient` to pull out a base class. ([\#15427](matrix-org/synapse#15427)) - Improve type hints. ([\#15432](matrix-org/synapse#15432)) - Convert async to normal tests in `TestSSOHandler`. ([\#15433](matrix-org/synapse#15433)) - Speed up the user directory background update. ([\#15435](matrix-org/synapse#15435)) - Disable directory listing for static resources in `/_matrix/static/`. ([\#15438](matrix-org/synapse#15438)) - Move various module API callback registration methods to a dedicated class. ([\#15453](matrix-org/synapse#15453)) Synapse 1.81.0 (2023-04-11) =========================== Synapse now attempts the versioned appservice paths before falling back to the [legacy paths](https://spec.matrix.org/v1.6/application-service-api/#legacy-routes). Usage of the legacy routes should be considered deprecated. Additionally, Synapse has supported sending the application service access token via [the `Authorization` header](https://spec.matrix.org/v1.6/application-service-api/#authorization) since v1.70.0. For backwards compatibility it is *also* sent as the `access_token` query parameter. This is insecure and should be considered deprecated. A future version of Synapse (v1.88.0 or later) will remove support for legacy application service routes and query parameter authorization. No significant changes since 1.81.0rc2. Synapse 1.81.0rc2 (2023-04-06) ============================== Bugfixes -------- - Fix the `set_device_id_for_pushers_txn` background update crash. ([\#15391](matrix-org/synapse#15391)) Internal Changes ---------------- - Update CI to run complement under the latest stable go version. ([\#15403](matrix-org/synapse#15403)) Synapse 1.81.0rc1 (2023-04-04) ============================== Features -------- - Add the ability to enable/disable registrations when in the OIDC flow. ([\#14978](matrix-org/synapse#14978)) - Add a primitive helper script for listing worker endpoints. ([\#15243](matrix-org/synapse#15243)) - Experimental support for passing One Time Key and device key requests to application services ([MSC3983](matrix-org/matrix-spec-proposals#3983) and [MSC3984](matrix-org/matrix-spec-proposals#3984)). ([\#15314](matrix-org/synapse#15314), [\#15321](matrix-org/synapse#15321)) - Allow loading `/password_policy` endpoint on workers. ([\#15331](matrix-org/synapse#15331)) - Add experimental support for Unix sockets. Contributed by Jason Little. ([\#15353](matrix-org/synapse#15353)) - Build Debian packages for Ubuntu 23.04 (Lunar Lobster). ([\#15381](matrix-org/synapse#15381)) Bugfixes -------- - Fix a long-standing bug where edits of non-`m.room.message` events would not be correctly bundled. ([\#15295](matrix-org/synapse#15295)) - Fix a bug introduced in Synapse v1.55.0 which could delay remote homeservers being able to decrypt encrypted messages sent by local users. ([\#15297](matrix-org/synapse#15297)) - Add a check to [SQLite port_db script](https://matrix-org.github.io/synapse/latest/postgres.html#porting-from-sqlite) to ensure that the sqlite database passed to the script exists before trying to port from it. ([\#15306](matrix-org/synapse#15306)) - Fix a bug introduced in Synapse 1.76.0 where responses from worker deployments could include an internal `_INT_STREAM_POS` key. ([\#15309](matrix-org/synapse#15309)) - Fix a long-standing bug that Synpase only used the [legacy appservice routes](https://spec.matrix.org/v1.6/application-service-api/#legacy-routes). ([\#15317](matrix-org/synapse#15317)) - Fix a long-standing bug preventing users from rejoining rooms after being banned and unbanned over federation. Contributed by Nico. ([\#15323](matrix-org/synapse#15323)) - Fix bug in worker mode where on a rolling restart of workers the "typing" worker would consume 100% CPU until it got restarted. ([\#15332](matrix-org/synapse#15332)) - Fix a long-standing bug where some to_device messages could be dropped when using workers. ([\#15349](matrix-org/synapse#15349)) - Fix a bug introduced in Synapse 1.70.0 where the background sync from a faster join could spin for hours when one of the events involved had been marked for backoff. ([\#15351](matrix-org/synapse#15351)) - Fix missing app variable in mail subject for password resets. Contributed by Cyberes. ([\#15352](matrix-org/synapse#15352)) - Fix a rare bug introduced in Synapse 1.66.0 where initial syncs would fail when the user had been kicked from a faster joined room that had not finished syncing. ([\#15383](matrix-org/synapse#15383)) Improved Documentation ---------------------- - Fix a typo in login requests ratelimit defaults. ([\#15341](matrix-org/synapse#15341)) - Add some clarification to the doc/comments regarding TCP replication. ([\#15354](matrix-org/synapse#15354)) - Note that Synapse 1.74 queued a rebuild of the user directory tables. ([\#15386](matrix-org/synapse#15386)) Internal Changes ---------------- - Use `immutabledict` instead of `frozendict`. ([\#15113](matrix-org/synapse#15113)) - Add developer documentation for the Federation Sender and add a documentation mechanism using Sphinx. ([\#15265](matrix-org/synapse#15265), [\#15336](matrix-org/synapse#15336)) - Make the pushers rely on the `device_id` instead of the `access_token_id` for various operations. ([\#15280](matrix-org/synapse#15280)) - Bump sentry-sdk from 1.15.0 to 1.17.0. ([\#15285](matrix-org/synapse#15285)) - Allow running the Twisted trunk job against other branches. ([\#15302](matrix-org/synapse#15302)) - Remind the releaser to ask for changelog feedback in [#synapse-dev](https://matrix.to/#/#synapse-dev:matrix.org). ([\#15303](matrix-org/synapse#15303)) - Bump dtolnay/rust-toolchain from e12eda571dc9a5ee5d58eecf4738ec291c66f295 to fc3253060d0c959bea12a59f10f8391454a0b02d. ([\#15304](matrix-org/synapse#15304)) - Reject events with an invalid "mentions" property per [MSC3952](matrix-org/matrix-spec-proposals#3952). ([\#15311](matrix-org/synapse#15311)) - As an optimisation, use `TRUNCATE` on Postgres when clearing the user directory tables. ([\#15316](matrix-org/synapse#15316)) - Fix `.gitignore` rule for the Complement source tarball downloaded automatically by `complement.sh`. ([\#15319](matrix-org/synapse#15319)) - Bump serde from 1.0.157 to 1.0.158. ([\#15324](matrix-org/synapse#15324)) - Bump regex from 1.7.1 to 1.7.3. ([\#15325](matrix-org/synapse#15325)) - Bump types-pyopenssl from 23.0.0.4 to 23.1.0.0. ([\#15326](matrix-org/synapse#15326)) - Bump furo from 2022.12.7 to 2023.3.23. ([\#15327](matrix-org/synapse#15327)) - Bump ruff from 0.0.252 to 0.0.259. ([\#15328](matrix-org/synapse#15328)) - Bump cryptography from 40.0.0 to 40.0.1. ([\#15329](matrix-org/synapse#15329)) - Bump mypy-zope from 0.9.0 to 0.9.1. ([\#15330](matrix-org/synapse#15330)) - Speed up unit tests when using SQLite3. ([\#15334](matrix-org/synapse#15334)) - Speed up pydantic CI job. ([\#15339](matrix-org/synapse#15339)) - Speed up sample config CI job. ([\#15340](matrix-org/synapse#15340)) - Fix copyright year in SSO footer template. ([\#15358](matrix-org/synapse#15358)) - Bump peaceiris/actions-gh-pages from 3.9.2 to 3.9.3. ([\#15369](matrix-org/synapse#15369)) - Bump serde from 1.0.158 to 1.0.159. ([\#15370](matrix-org/synapse#15370)) - Bump serde_json from 1.0.94 to 1.0.95. ([\#15371](matrix-org/synapse#15371)) - Speed up membership queries for users with forgotten rooms. ([\#15385](matrix-org/synapse#15385))

XxSNiPxX · 2024-06-02T11:12:04Z

Hello guys,

My current project requires me to authenticate a user on my synapse instance via a signed message from a keplr crypto wallet . I have a wallet signed message used as the password. How would i approach the authentication on matrix? Would i use another server that is openid compatible, whose license is often not gplv3 or would i use the architecture before by contacting the synapse instance directly? Can someone please share and end to end example of setting up a IAM server between the client and synapse?

Thanks

@Michael-Hollister

# Synapse 1.112.0 (2024-07-30) This security release is to update our locked dependency on Twisted to 24.7.0rc1, which includes a security fix for [CVE-2024-41671 / GHSA-c8m8-j448-xjx7: Disordered HTTP pipeline response in twisted.web, again](GHSA-c8m8-j448-xjx7). Note that this security fix is also available as **Synapse 1.111.1**, which does not include the rest of the changes in Synapse 1.112.0. This issue means that, if multiple HTTP requests are pipelined in the same TCP connection, Synapse can send responses to the wrong HTTP request. If a reverse proxy was configured to use HTTP pipelining, this could result in responses being sent to the wrong user, severely harming confidentiality. With that said, despite being a high severity issue, **we consider it unlikely that Synapse installations will be affected**. The use of HTTP pipelining in this fashion would cause worse performance for clients (request-response latencies would be increased as users' responses would be artificially blocked behind other users' slow requests). Further, Nginx and Haproxy, two common reverse proxies, do not appear to support configuring their upstreams to use HTTP pipelining and thus would not be affected. For both of these reasons, we consider it unlikely that a Synapse deployment would be set up in such a configuration. Despite that, we cannot rule out that some installations may exist with this unusual setup and so we are releasing this security update today. **pip users:** Note that by default, upgrading Synapse using pip will not automatically upgrade Twisted. **Please manually install the new version of Twisted** using `pip install Twisted==24.7.0rc1`. Note also that even the `--upgrade-strategy=eager` flag to `pip install -U matrix-synapse` will not upgrade Twisted to a patched version because it is only a release candidate at this time. ### Internal Changes - Upgrade locked dependency on Twisted to 24.7.0rc1. ([\#17502](element-hq/synapse#17502)) # Synapse 1.112.0rc1 (2024-07-23) Please note that this release candidate does not include the security dependency update included in version 1.111.1 as this version was released before 1.111.1. The same security fix can be found in the full release of 1.112.0. ### Features - Add to-device extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17416](element-hq/synapse#17416)) - Populate `name`/`avatar` fields in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17418](element-hq/synapse#17418)) - Populate `heroes` and room summary fields (`joined_count`, `invited_count`) in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17419](element-hq/synapse#17419)) - Populate `is_dm` room field in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17429](element-hq/synapse#17429)) - Add room subscriptions to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17432](element-hq/synapse#17432)) - Prepare for authenticated media freeze. ([\#17433](element-hq/synapse#17433)) - Add E2EE extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17454](element-hq/synapse#17454)) ### Bugfixes - Add configurable option to always include offline users in presence sync results. Contributed by @Michael-Hollister. ([\#17231](element-hq/synapse#17231)) - Fix bug in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint when using room type filters and the user has one or more remote invites. ([\#17434](element-hq/synapse#17434)) - Order `heroes` by `stream_ordering` as the Matrix specification states (applies to `/sync`). ([\#17435](element-hq/synapse#17435)) - Fix rare bug where `/sync` would break for a user when using workers with multiple stream writers. ([\#17438](element-hq/synapse#17438)) ### Improved Documentation - Update the readme image to have a white background, so that it is readable in dark mode. ([\#17387](element-hq/synapse#17387)) - Add Red Hat Enterprise Linux and Rocky Linux 8 and 9 installation instructions. ([\#17423](element-hq/synapse#17423)) - Improve documentation for the [`default_power_level_content_override`](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#default_power_level_content_override) config option. ([\#17451](element-hq/synapse#17451)) ### Internal Changes - Make sure we always use the right logic for enabling the media repo. ([\#17424](element-hq/synapse#17424)) - Fix argument documentation for method `RateLimiter.record_action`. ([\#17426](element-hq/synapse#17426)) - Reduce volume of 'Waiting for current token' logs, which were introduced in v1.109.0. ([\#17428](element-hq/synapse#17428)) - Limit concurrent remote downloads to 6 per IP address, and decrement remote downloads without a content-length from the ratelimiter after the download is complete. ([\#17439](element-hq/synapse#17439)) - Remove unnecessary call to resume producing in fake channel. ([\#17449](element-hq/synapse#17449)) - Update experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint to bump room when it is created. ([\#17453](element-hq/synapse#17453)) - Speed up generating sliding sync responses. ([\#17458](element-hq/synapse#17458)) - Add cache to `get_rooms_for_local_user_where_membership_is` to speed up sliding sync. ([\#17460](element-hq/synapse#17460)) - Speed up fetching room keys from backup. ([\#17461](element-hq/synapse#17461)) - Speed up sorting of the room list in sliding sync. ([\#17468](element-hq/synapse#17468)) - Implement handling of `$ME` as a state key in sliding sync. ([\#17469](element-hq/synapse#17469)) ### Updates to locked dependencies * Bump bytes from 1.6.0 to 1.6.1. ([\#17441](element-hq/synapse#17441)) * Bump hiredis from 2.3.2 to 3.0.0. ([\#17464](element-hq/synapse#17464)) * Bump jsonschema from 4.22.0 to 4.23.0. ([\#17444](element-hq/synapse#17444)) * Bump matrix-org/done-action from 2 to 3. ([\#17440](element-hq/synapse#17440)) * Bump mypy from 1.9.0 to 1.10.1. ([\#17445](element-hq/synapse#17445)) * Bump pyopenssl from 24.1.0 to 24.2.1. ([\#17465](element-hq/synapse#17465)) * Bump ruff from 0.5.0 to 0.5.4. ([\#17466](element-hq/synapse#17466)) * Bump sentry-sdk from 2.6.0 to 2.8.0. ([\#17456](element-hq/synapse#17456)) * Bump sentry-sdk from 2.8.0 to 2.10.0. ([\#17467](element-hq/synapse#17467)) * Bump setuptools from 67.6.0 to 70.0.0. ([\#17448](element-hq/synapse#17448)) * Bump twine from 5.1.0 to 5.1.1. ([\#17443](element-hq/synapse#17443)) * Bump types-jsonschema from 4.22.0.20240610 to 4.23.0.20240712. ([\#17446](element-hq/synapse#17446)) * Bump ulid from 1.1.2 to 1.1.3. ([\#17442](element-hq/synapse#17442)) * Bump zipp from 3.15.0 to 3.19.1. ([\#17427](element-hq/synapse#17427)) # Synapse 1.111.0 (2024-07-16) No significant changes since 1.111.0rc2. # Synapse 1.111.0rc2 (2024-07-10) ### Bugfixes - Fix bug where using `synapse.app.media_repository` worker configuration would break the new media endpoints. ([\#17420](element-hq/synapse#17420)) ### Improved Documentation - Document the new federation media worker endpoints in the [upgrade notes](https://element-hq.github.io/synapse/v1.111/upgrade.html) and [worker docs](https://element-hq.github.io/synapse/v1.111/workers.html). ([\#17421](element-hq/synapse#17421)) ### Internal Changes - Route authenticated federation media requests to media repository workers in Complement tests. ([\#17422](element-hq/synapse#17422)) # Synapse 1.111.0rc1 (2024-07-09) ### Features - Add `rooms` data to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17320](element-hq/synapse#17320)) - Add `room_types`/`not_room_types` filtering to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17337](element-hq/synapse#17337)) - Return "required state" in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17342](element-hq/synapse#17342)) - Support [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/3916-authentication-for-media.md) by adding [`_matrix/client/v1/media/download`](https://spec.matrix.org/v1.11/client-server-api/#get_matrixclientv1mediadownloadservernamemediaid) endpoint. ([\#17365](element-hq/synapse#17365)) - Support [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/authentication-for-media/proposals/3916-authentication-for-media.md) by adding [`_matrix/client/v1/media/thumbnail`](https://spec.matrix.org/v1.11/client-server-api/#get_matrixclientv1mediathumbnailservernamemediaid), [`_matrix/federation/v1/media/thumbnail`](https://spec.matrix.org/v1.11/server-server-api/#get_matrixfederationv1mediathumbnailmediaid) endpoints and stabilizing the remaining [`_matrix/client/v1/media`](https://spec.matrix.org/v1.11/client-server-api/#get_matrixclientv1mediaconfig) endpoints. ([\#17388](element-hq/synapse#17388)) - Add `rooms.bump_stamp` for easier client-side sorting in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17395](element-hq/synapse#17395)) - Forget all of a user's rooms upon deactivation, preventing local room purges from being blocked on deactivated users. ([\#17400](element-hq/synapse#17400)) - Declare support for [Matrix 1.11](https://matrix.org/blog/2024/06/20/matrix-v1.11-release/). ([\#17403](element-hq/synapse#17403)) - [MSC3861](matrix-org/matrix-spec-proposals#3861): allow overriding the introspection endpoint. ([\#17406](element-hq/synapse#17406)) ### Bugfixes - Fix rare race which caused no new to-device messages to be received from remote server. ([\#17362](element-hq/synapse#17362)) - Fix bug in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint when using an old database. ([\#17398](element-hq/synapse#17398)) ### Improved Documentation - Clarify that `url_preview_url_blacklist` is a usability feature. ([\#17356](element-hq/synapse#17356)) - Fix broken links in README. ([\#17379](element-hq/synapse#17379)) - Clarify that changelog content *and file extension* need to match in order for entries to merge. ([\#17399](element-hq/synapse#17399)) ### Internal Changes - Make the release script create a release branch for Complement as well. ([\#17318](element-hq/synapse#17318)) - Fix uploading packages to PyPi. ([\#17363](element-hq/synapse#17363)) - Add CI check for the README. ([\#17367](element-hq/synapse#17367)) - Fix linting errors from new `ruff` version. ([\#17381](element-hq/synapse#17381), [\#17411](element-hq/synapse#17411)) - Fix building debian packages on non-clean checkouts. ([\#17390](element-hq/synapse#17390)) - Finish up work to allow per-user feature flags. ([\#17392](element-hq/synapse#17392), [\#17410](element-hq/synapse#17410)) - Allow enabling sliding sync per-user. ([\#17393](element-hq/synapse#17393)) ### Updates to locked dependencies * Bump certifi from 2023.7.22 to 2024.7.4. ([\#17404](element-hq/synapse#17404)) * Bump cryptography from 42.0.7 to 42.0.8. ([\#17382](element-hq/synapse#17382)) * Bump ijson from 3.2.3 to 3.3.0. ([\#17413](element-hq/synapse#17413)) * Bump log from 0.4.21 to 0.4.22. ([\#17384](element-hq/synapse#17384)) * Bump mypy-zope from 1.0.4 to 1.0.5. ([\#17414](element-hq/synapse#17414)) * Bump pillow from 10.3.0 to 10.4.0. ([\#17412](element-hq/synapse#17412)) * Bump pydantic from 2.7.1 to 2.8.2. ([\#17415](element-hq/synapse#17415)) * Bump ruff from 0.3.7 to 0.5.0. ([\#17381](element-hq/synapse#17381)) * Bump serde from 1.0.203 to 1.0.204. ([\#17409](element-hq/synapse#17409)) * Bump serde_json from 1.0.117 to 1.0.120. ([\#17385](element-hq/synapse#17385), [\#17408](element-hq/synapse#17408)) * Bump types-setuptools from 69.5.0.20240423 to 70.1.0.20240627. ([\#17380](element-hq/synapse#17380)) # Synapse 1.110.0 (2024-07-03) No significant changes since 1.110.0rc3. # Synapse 1.110.0rc3 (2024-07-02) ### Bugfixes - Fix bug where `/sync` requests could get blocked indefinitely after an upgrade from Synapse versions before v1.109.0. ([\#17386](element-hq/synapse#17386), [\#17391](element-hq/synapse#17391)) ### Internal Changes - Limit size of presence EDUs to 50 entries. ([\#17371](element-hq/synapse#17371)) - Fix building debian package for debian sid. ([\#17389](element-hq/synapse#17389)) # Synapse 1.110.0rc2 (2024-06-26) ### Internal Changes - Fix uploading packages to PyPi. ([\#17363](element-hq/synapse#17363)) # Synapse 1.110.0rc1 (2024-06-26) ### Features - Add initial implementation of an experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17187](element-hq/synapse#17187)) - Add experimental support for [MSC3823](matrix-org/matrix-spec-proposals#3823) - Account suspension. ([\#17255](element-hq/synapse#17255)) - Improve ratelimiting in Synapse. ([\#17256](element-hq/synapse#17256)) - Add support for the unstable [MSC4151](matrix-org/matrix-spec-proposals#4151) report room API. ([\#17270](element-hq/synapse#17270), [\#17296](element-hq/synapse#17296)) - Filter for public and empty rooms added to Admin-API [List Room API](https://element-hq.github.io/synapse/latest/admin_api/rooms.html#list-room-api). ([\#17276](element-hq/synapse#17276)) - Add `is_dm` filtering to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17277](element-hq/synapse#17277)) - Add `is_encrypted` filtering to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17281](element-hq/synapse#17281)) - Include user membership in events served to clients, per [MSC4115](matrix-org/matrix-spec-proposals#4115). ([\#17282](element-hq/synapse#17282)) - Do not require user-interactive authentication for uploading cross-signing keys for the first time, per [MSC3967](matrix-org/matrix-spec-proposals#3967). ([\#17284](element-hq/synapse#17284)) - Add `stream_ordering` sort to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17293](element-hq/synapse#17293)) - `register_new_matrix_user` now supports a --password-file flag, which is useful for scripting. ([\#17294](element-hq/synapse#17294)) - `register_new_matrix_user` now supports a --exists-ok flag to allow registration of users that already exist in the database. This is useful for scripts that bootstrap user accounts with initial passwords. ([\#17304](element-hq/synapse#17304)) - Add support for via query parameter from [MSC4156](matrix-org/matrix-spec-proposals#4156). ([\#17322](element-hq/synapse#17322)) - Add `is_invite` filtering to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17335](element-hq/synapse#17335)) - Support [MSC3916](https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/3916-authentication-for-media.md) by adding a federation /download endpoint. ([\#17350](element-hq/synapse#17350)) ### Bugfixes - Fix searching for users with their exact localpart whose ID includes a hyphen. ([\#17254](element-hq/synapse#17254)) - Fix wrong retention policy being used when filtering events. ([\#17272](element-hq/synapse#17272)) - Fix bug where OTKs were not always included in `/sync` response when using workers. ([\#17275](element-hq/synapse#17275)) - Fix a long-standing bug where an invalid 'from' parameter to [`/notifications`](https://spec.matrix.org/v1.10/client-server-api/#get_matrixclientv3notifications) would result in an Internal Server Error. ([\#17283](element-hq/synapse#17283)) - Fix edge case in `/sync` returning the wrong the state when using sharded event persisters. ([\#17295](element-hq/synapse#17295)) - Add initial implementation of an experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17301](element-hq/synapse#17301)) - Fix email notification subject when invited to a space. ([\#17336](element-hq/synapse#17336)) ### Improved Documentation - Add missing quotes for example for `exclude_rooms_from_sync`. ([\#17308](element-hq/synapse#17308)) - Update header in the README to visually fix the the auto-generated table of contents. ([\#17329](element-hq/synapse#17329)) - Fix stale references to the Foundation's Security Disclosure Policy. ([\#17341](element-hq/synapse#17341)) - Add default values for `rc_invites.per_issuer` to docs. ([\#17347](element-hq/synapse#17347)) - Fix an error in the docs for `search_all_users` parameter under `user_directory`. ([\#17348](element-hq/synapse#17348)) ### Internal Changes - Remove unused `expire_access_token` option in the Synapse Docker config file. Contributed by @AaronDewes. ([\#17198](element-hq/synapse#17198)) - Use fully-qualified `PersistedEventPosition` when returning `RoomsForUser` to facilitate proper comparisons and `RoomStreamToken` generation. ([\#17265](element-hq/synapse#17265)) - Add debug logging for when room keys are uploaded, including whether they are replacing other room keys. ([\#17266](element-hq/synapse#17266)) - Handle OTK uploads off master. ([\#17271](element-hq/synapse#17271)) - Don't try and resync devices for remote users whose servers are marked as down. ([\#17273](element-hq/synapse#17273)) - Re-organize Pydantic models and types used in handlers. ([\#17279](element-hq/synapse#17279)) - Expose the worker instance that persisted the event on `event.internal_metadata.instance_name`. ([\#17300](element-hq/synapse#17300)) - Update the README with Element branding, improve headers and fix the #synapse:matrix.org support room link rendering. ([\#17324](element-hq/synapse#17324)) - Change path of the experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync implementation to `/org.matrix.simplified_msc3575/sync` since our simplified API is slightly incompatible with what's in the current MSC. ([\#17331](element-hq/synapse#17331)) - Handle device lists notifications for large accounts more efficiently in worker mode. ([\#17333](element-hq/synapse#17333), [\#17358](element-hq/synapse#17358)) - Do not block event sending/receiving while calculating large event auth chains. ([\#17338](element-hq/synapse#17338)) - Tidy up `parse_integer` docs and call sites to reflect the fact that they require non-negative integers by default, and bring `parse_integer_from_args` default in alignment. Contributed by Denis Kasak (@dkasak). ([\#17339](element-hq/synapse#17339)) ### Updates to locked dependencies * Bump authlib from 1.3.0 to 1.3.1. ([\#17343](element-hq/synapse#17343)) * Bump dawidd6/action-download-artifact from 3.1.4 to 5. ([\#17289](element-hq/synapse#17289)) * Bump dawidd6/action-download-artifact from 5 to 6. ([\#17313](element-hq/synapse#17313)) * Bump docker/build-push-action from 5 to 6. ([\#17312](element-hq/synapse#17312)) * Bump jinja2 from 3.1.3 to 3.1.4. ([\#17287](element-hq/synapse#17287)) * Bump lazy_static from 1.4.0 to 1.5.0. ([\#17355](element-hq/synapse#17355)) * Bump msgpack from 1.0.7 to 1.0.8. ([\#17317](element-hq/synapse#17317)) * Bump netaddr from 1.2.1 to 1.3.0. ([\#17353](element-hq/synapse#17353)) * Bump packaging from 24.0 to 24.1. ([\#17352](element-hq/synapse#17352)) * Bump phonenumbers from 8.13.37 to 8.13.39. ([\#17315](element-hq/synapse#17315)) * Bump regex from 1.10.4 to 1.10.5. ([\#17290](element-hq/synapse#17290)) * Bump requests from 2.31.0 to 2.32.2. ([\#17345](element-hq/synapse#17345)) * Bump sentry-sdk from 2.1.1 to 2.3.1. ([\#17263](element-hq/synapse#17263)) * Bump sentry-sdk from 2.3.1 to 2.6.0. ([\#17351](element-hq/synapse#17351)) * Bump tornado from 6.4 to 6.4.1. ([\#17344](element-hq/synapse#17344)) * Bump mypy from 1.8.0 to 1.9.0. ([\#17297](element-hq/synapse#17297)) * Bump types-jsonschema from 4.21.0.20240311 to 4.22.0.20240610. ([\#17288](element-hq/synapse#17288)) * Bump types-netaddr from 1.2.0.20240219 to 1.3.0.20240530. ([\#17314](element-hq/synapse#17314)) * Bump types-pillow from 10.2.0.20240423 to 10.2.0.20240520. ([\#17285](element-hq/synapse#17285)) * Bump types-pyyaml from 6.0.12.12 to 6.0.12.20240311. ([\#17316](element-hq/synapse#17316)) * Bump typing-extensions from 4.11.0 to 4.12.2. ([\#17354](element-hq/synapse#17354)) * Bump urllib3 from 2.0.7 to 2.2.2. ([\#17346](element-hq/synapse#17346))

@kfiven

# Synapse 1.114.0 (2024-09-02) This release enables support for [MSC4186](matrix-org/matrix-spec-proposals#4186) — Simplified Sliding Sync. This allows using the upcoming releases of the Element X mobile apps without having to run a Sliding Sync Proxy. ### Features - Enable native sliding sync support ([MSC3575](matrix-org/matrix-spec-proposals#3575) and [MSC4186](matrix-org/matrix-spec-proposals#4186)) by default. ([\#17648](element-hq/synapse#17648)) # Synapse 1.114.0rc3 (2024-08-30) ### Bugfixes - Fix regression in v1.114.0rc2 that caused workers to fail to start. ([\#17626](element-hq/synapse#17626)) # Synapse 1.114.0rc2 (2024-08-30) ### Features - Improve cross-signing upload when using [MSC3861](matrix-org/matrix-spec-proposals#3861) to use a custom UIA flow stage, with web fallback support. ([\#17509](element-hq/synapse#17509)) - Make `hash_password` script accept password input from stdin. ([\#17608](element-hq/synapse#17608)) ### Bugfixes - Fix hierarchy returning 403 when room is accessible through federation. Contributed by Krishan (@kfiven). ([\#17194](element-hq/synapse#17194)) - Fix content-length on federation `/thumbnail` responses. ([\#17532](element-hq/synapse#17532)) - Fix authenticated media responses using a wrong limit when following redirects over federation. ([\#17543](element-hq/synapse#17543)) ### Internal Changes - MSC3861: load the issuer and account management URLs from OIDC discovery. ([\#17407](element-hq/synapse#17407)) - Refactor sliding sync class into multiple files. ([\#17595](element-hq/synapse#17595)) - Store sliding sync per-connection state in the database. ([\#17599](element-hq/synapse#17599)) - Make the sliding sync `PerConnectionState` class immutable. ([\#17600](element-hq/synapse#17600)) - Add support to `@tag_args` for standalone functions. ([\#17604](element-hq/synapse#17604)) - Speed up incremental syncs in sliding sync by adding some more caching. ([\#17606](element-hq/synapse#17606)) - Always return the user's own read receipts in sliding sync. ([\#17617](element-hq/synapse#17617)) - Replace `isort` and `black` with `ruff`. ([\#17620](element-hq/synapse#17620)) - Refactor sliding sync code to move room list logic out into a separate class. ([\#17622](element-hq/synapse#17622)) ### Updates to locked dependencies * Bump attrs from 23.2.0 to 24.2.0. ([\#17609](element-hq/synapse#17609)) * Bump cryptography from 42.0.8 to 43.0.0. ([\#17584](element-hq/synapse#17584)) * Bump phonenumbers from 8.13.43 to 8.13.44. ([\#17610](element-hq/synapse#17610)) * Bump pygithub from 2.3.0 to 2.4.0. ([\#17612](element-hq/synapse#17612)) * Bump pyyaml from 6.0.1 to 6.0.2. ([\#17611](element-hq/synapse#17611)) * Bump sentry-sdk from 2.12.0 to 2.13.0. ([\#17585](element-hq/synapse#17585)) * Bump serde from 1.0.206 to 1.0.208. ([\#17581](element-hq/synapse#17581)) * Bump serde from 1.0.208 to 1.0.209. ([\#17613](element-hq/synapse#17613)) * Bump serde_json from 1.0.124 to 1.0.125. ([\#17582](element-hq/synapse#17582)) * Bump serde_json from 1.0.125 to 1.0.127. ([\#17614](element-hq/synapse#17614)) * Bump types-jsonschema from 4.23.0.20240712 to 4.23.0.20240813. ([\#17583](element-hq/synapse#17583)) * Bump types-setuptools from 71.1.0.20240726 to 71.1.0.20240818. ([\#17586](element-hq/synapse#17586)) # Synapse 1.114.0rc1 (2024-08-20) ### Features - Add a flag to `/versions`, `org.matrix.simplified_msc3575`, to indicate whether experimental sliding sync support has been enabled. ([\#17571](element-hq/synapse#17571)) - Handle changes in `timeline_limit` in experimental sliding sync. ([\#17579](element-hq/synapse#17579)) - Correctly track read receipts that should be sent down in experimental sliding sync. ([\#17575](element-hq/synapse#17575), [\#17589](element-hq/synapse#17589), [\#17592](element-hq/synapse#17592)) ### Bugfixes - Start handlers for new media endpoints when media resource configured. ([\#17483](element-hq/synapse#17483)) - Fix timeline ordering (using `stream_ordering` instead of topological ordering) in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17510](element-hq/synapse#17510)) - Fix experimental sliding sync implementation to remember any updates in rooms that were not sent down immediately. ([\#17535](element-hq/synapse#17535)) - Better exclude partially stated rooms if we must await full state in experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17538](element-hq/synapse#17538)) - Handle lower-case http headers in `_Mulitpart_Parser_Protocol`. ([\#17545](element-hq/synapse#17545)) - Fix fetching federation signing keys from servers that omit `old_verify_keys`. Contributed by @tulir @ Beeper. ([\#17568](element-hq/synapse#17568)) - Fix bug where we would respond with an error when a remote server asked for media that had a length of 0, using the new multipart federation media endpoint. ([\#17570](element-hq/synapse#17570)) ### Improved Documentation - Clarify default behaviour of the [`auto_accept_invites.worker_to_run_on`](https://element-hq.github.io/synapse/develop/usage/configuration/config_documentation.html#auto-accept-invites) option. ([\#17515](element-hq/synapse#17515)) - Improve docstrings for profile methods. ([\#17559](element-hq/synapse#17559)) ### Internal Changes - Add more tracing to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17514](element-hq/synapse#17514)) - Fixup comment in sliding sync implementation. ([\#17531](element-hq/synapse#17531)) - Replace override of deprecated method `HTTPAdapter.get_connection` with `get_connection_with_tls_context`. ([\#17536](element-hq/synapse#17536)) - Fix performance of device lists in `/key/changes` and sliding sync. ([\#17537](element-hq/synapse#17537), [\#17548](element-hq/synapse#17548)) - Bump setuptools from 67.6.0 to 72.1.0. ([\#17542](element-hq/synapse#17542)) - Add a utility function for generating random event IDs. ([\#17557](element-hq/synapse#17557)) - Speed up responding to media requests. ([\#17558](element-hq/synapse#17558), [\#17561](element-hq/synapse#17561), [\#17564](element-hq/synapse#17564), [\#17566](element-hq/synapse#17566), [\#17567](element-hq/synapse#17567), [\#17569](element-hq/synapse#17569)) - Test github token before running release script steps. ([\#17562](element-hq/synapse#17562)) - Reduce log spam of multipart files. ([\#17563](element-hq/synapse#17563)) - Refactor per-connection state in experimental sliding sync handler. ([\#17574](element-hq/synapse#17574)) - Add histogram metrics for sliding sync processing time. ([\#17593](element-hq/synapse#17593)) ### Updates to locked dependencies * Bump bytes from 1.6.1 to 1.7.1. ([\#17526](element-hq/synapse#17526)) * Bump lxml from 5.2.2 to 5.3.0. ([\#17550](element-hq/synapse#17550)) * Bump phonenumbers from 8.13.42 to 8.13.43. ([\#17551](element-hq/synapse#17551)) * Bump regex from 1.10.5 to 1.10.6. ([\#17527](element-hq/synapse#17527)) * Bump sentry-sdk from 2.10.0 to 2.12.0. ([\#17553](element-hq/synapse#17553)) * Bump serde from 1.0.204 to 1.0.206. ([\#17556](element-hq/synapse#17556)) * Bump serde_json from 1.0.122 to 1.0.124. ([\#17555](element-hq/synapse#17555)) * Bump sigstore/cosign-installer from 3.5.0 to 3.6.0. ([\#17549](element-hq/synapse#17549)) * Bump types-pyyaml from 6.0.12.20240311 to 6.0.12.20240808. ([\#17552](element-hq/synapse#17552)) * Bump types-requests from 2.31.0.20240406 to 2.32.0.20240712. ([\#17524](element-hq/synapse#17524)) # Synapse 1.113.0 (2024-08-13) No significant changes since 1.113.0rc1. # Synapse 1.113.0rc1 (2024-08-06) ### Features - Track which rooms have been sent to clients in the experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17447](element-hq/synapse#17447)) - Add Account Data extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17477](element-hq/synapse#17477)) - Add receipts extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17489](element-hq/synapse#17489)) - Add typing notification extension support to experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint. ([\#17505](element-hq/synapse#17505)) ### Bugfixes - Update experimental [MSC3575](matrix-org/matrix-spec-proposals#3575) Sliding Sync `/sync` endpoint to handle invite/knock rooms when filtering. ([\#17450](element-hq/synapse#17450)) - Fix a bug introduced in v1.110.0 which caused `/keys/query` to return incomplete results, leading to high network activity and CPU usage on Matrix clients. ([\#17499](element-hq/synapse#17499)) ### Improved Documentation - Update the [`allowed_local_3pids`](https://element-hq.github.io/synapse/v1.112/usage/configuration/config_documentation.html#allowed_local_3pids) config option's msisdn address to a working example. ([\#17476](element-hq/synapse#17476)) ### Internal Changes - Change sliding sync to use their own token format in preparation for storing per-connection state. ([\#17452](element-hq/synapse#17452)) - Ensure we don't send down negative `bump_stamp` in experimental sliding sync endpoint. ([\#17478](element-hq/synapse#17478)) - Do not send down empty room entries down experimental sliding sync endpoint. ([\#17479](element-hq/synapse#17479)) - Refactor Sliding Sync tests to better utilize the `SlidingSyncBase`. ([\#17481](element-hq/synapse#17481), [\#17482](element-hq/synapse#17482)) - Add some opentracing tags and logging to the experimental sliding sync implementation. ([\#17501](element-hq/synapse#17501)) - Split and move Sliding Sync tests so we have some more sane test file sizes. ([\#17504](element-hq/synapse#17504)) - Update the `limited` field description in the Sliding Sync response to accurately describe what it actually represents. ([\#17507](element-hq/synapse#17507)) - Easier to understand `timeline` assertions in Sliding Sync tests. ([\#17511](element-hq/synapse#17511)) - Reset the sliding sync connection if we don't recognize the per-connection state position. ([\#17529](element-hq/synapse#17529)) ### Updates to locked dependencies * Bump bcrypt from 4.1.3 to 4.2.0. ([\#17495](element-hq/synapse#17495)) * Bump black from 24.4.2 to 24.8.0. ([\#17522](element-hq/synapse#17522)) * Bump phonenumbers from 8.13.39 to 8.13.42. ([\#17521](element-hq/synapse#17521)) * Bump ruff from 0.5.4 to 0.5.5. ([\#17494](element-hq/synapse#17494)) * Bump serde_json from 1.0.120 to 1.0.121. ([\#17493](element-hq/synapse#17493)) * Bump serde_json from 1.0.121 to 1.0.122. ([\#17525](element-hq/synapse#17525)) * Bump towncrier from 23.11.0 to 24.7.1. ([\#17523](element-hq/synapse#17523)) * Bump types-pyopenssl from 24.1.0.20240425 to 24.1.0.20240722. ([\#17496](element-hq/synapse#17496)) * Bump types-setuptools from 70.1.0.20240627 to 71.1.0.20240726. ([\#17497](element-hq/synapse#17497))

Xiretza · 2024-09-16T17:37:25Z

proposals/3861-delegated-oidc-architecture.md

+- When setting up a CAPTCHA challenge, CAPTCHA services expect the challenge to be served from a specific domain. Because the client can be on any domain, [Synapse currently advises disabling host verification](https://element-hq.github.io/synapse/latest/CAPTCHA_SETUP.html#getting-api-keys).
+  When this option is disabled, the CAPTCHA service expects the server to verify the domain of the challenge, which is not possible.
+- The agreement to the terms of service can be completely ignored by the client, as it only requires the client to send a request with no specific data to validate it.
+  This means the server has to trust the goodwill of the client to make sure the user agrees to the terms of service.
+- The current design of the email verification flow has a confusing user experience.
+  The link sent to the user by email leads to a screen where the user is prompted to return to their client.


The first three of these points may be relevant for large, open-registration servers like matrix.org, but I'm gonna go out on a limb and say that most servers out there are not that. In general, there needs to be extremely solid justification for forcing this increase in complexity onto all the servers that have nothing or very little to gain from it.

This section talks about one particular example, which is also the homeserver where a lot of users will first experience this flow.

I'm not sure I understand the second part of your comment, how is this section talking about increasing the complexity onto all servers?

I'm not sure I understand the second part of your comment, how is this section talking about increasing the complexity onto all servers?

It's not about that section, which is why I wrote "in general" - the rewritten MSC later mentions:

The long-term goal is to deprecate the existing UIA APIs and replace them with the new OAuth 2.0/OIDC-based APIs.

This means that in the long term, all homeserver operators have to set up an entire OAuth/OIDC infrastructure, which is significantly more complex than the built-in username+password auth that is entirely sufficient for what I claim to be the vast majority of homeservers out there. Or am I misunderstanding something?

The change is relatively transparent for server operators, it is an API change, not an architecture change.

It happens that the current implementation with matrix-authentication-service and Synapse is a separate service, but the long term goal is to be integrated just within synapse, so the change will be fairly transparent for server operators

Does that answer your concern?

the long term goal is to be integrated just within synapse, so the change will be fairly transparent for server operators

Synapse operators. What about other homeservers?

Actually, putting on my homeserver developer hat, this really only shifts the problem from operators to developers. Right now, a homeserver implementation that can cover most usecases (i.e. except for the "large, open-registration" niche) using a very simple, almost stub-like UIA implementation. I admit I haven't actually looked into it properly yet, but I suspect that OIDC will make even a minimal implementation much more complex.

I suggest reading the rest of the proposal, as this is not about imposing stuff like email verification, CAPTCHA support, etc. on homeservers.

A very lightweight implementation of this can be done only supporting simple login and password. At a very high level, the new API look like this:

the client sends its metadata (name, redirect URL, etc.) to the homeserver, gets back a client identifier

the client redirects the user to a web page hosted by the homeserver

the homeserver does whatever it needs on that homepage to authenticate the user, this can be a very simple HTML form

once completed, the homeserver redirect backs to the client with single-use code

the client exchanges this code to get back an access token

How does that feel in terms of complexity? I understand that it is substantially higher than just sending 'username and password' to an API and get back an access token, but the bar doesn't seem too high for new implementation, especially considering there are plenty of OAuth/OIDC related libraries in many ecosystems to help implement some of this.

One use case I have somewhat frequently is setting up bots. Currently, you can just type in username+password, it does the authentication and stores the access token somewhere. With OIDC, I'd already be stuck at the first step: what is the "redirect URL" for a bot running on a headless server somewhere? Step 2, "redirecting" the user to a login page, is still possible by making them click on a link, but this is then opened on the user's machine, which is totally separate from the machine running the bot. There would need to be a third service somewhere that displays the single-use code, which then needs to be copy-pasted back into the bot. Not only is this more annoying to implement, it's also much more complicated to use.

So no, I'm not on board with removing username+password auth, but how about this as a long term plan:

Strip down UIA to a bare minimum username+password (this removes most of the complexity there and still covers most use cases),

Optionally let servers force the use of OIDC, which can then include all the captcha/email/whathaveyou bells and whistles for complex installations without having to cover these complexities in the Matrix spec.

Homeservers which implement the minimum interface can also create exceptions for bots as needed. It's a common practice for API tokens to expire and need rotation, and it's equally common that service providers have a way to mint tokens independent of UI flows. Take for example GitHub, which supports OAuth2/OIDC for applications to log in as the user and a healthy ecosystem of integrations which run headless - the integrations get a dedicated API token, which may expire, and is created within an OIDC-authenticated client. Something similar would likely be possible for homeserver admins (and from discussions I've seen, very much in the plan for consideration when speaking towards removal of UIA from the spec - bots would be a blocker before we remove UIA).

It's also considered bad common practice to store passwords on bots for the purposes of authentication, and we already see bots which implement this behaviour poorly. For example, logging in every time they need to make a request. With a specific minted token, they aren't placing burden on their homeservers/federation, and have a higher default security profile. This does add some code to bots/clients though, but may be able to be packaged away into a library or their underlying Matrix SDK of choice. It'd be the specification's role to (if needed) define a standard way to perform that authentication so those Matrix SDKs can consistently implement it.

Overall, OIDC retains the ability for the homeserver admin to choose what authentication mechanisms they want to support, while also giving them even more options. Matrix is naturally slower at adding things like 2FA, Passkeys, API tokens, scoped tokens, etc and we're not experts in it - this MSC acknowledges this gap and suggests we adopt a protocol which specializes in this. Homeserver developers can choose to bundle in a full-blown OIDC server in their install if they like, or they could implement more usecase-specific APIs, or somewhere inbetween. Synapse appears to be going a route closer to a full-blown OIDC server, largely because of the wide variety of usecases they need to support.

sandhose

The long-awaited big rewrite is here!

I've been working on making this series of MSCs a lot better, and I think I addressed a lot of concerns made on it. I commented on most open threads how I believe they are now addressed and closed them. If you believe those concerns are still not addressed, please reopen them or open new threads on the new MSC text

Happy reading!

proposals/3861-delegated-oidc-architecture.md

sandhose · 2024-09-17T08:57:46Z

proposals/3861-delegated-oidc-architecture.md

+
+## Security considerations
+
+Please refer to individual proposals.


I'm not sure how to properly address this to be honest. MSC2966 outlines what the client must advertise, and what the homeserver should accept. If the HS doesn't accept a valid registration, I would argue that this HS is no longer Matrix-compliant on the C-S API side?

This was always a possibility in the ecosystem: nothing prevented a homeserver to have an allow-list of valid redirect URLs for the m.login.sso flow.
What this proposal change, is that the homeserver has more metadata on the client than before to work with, which is a good thing for giving homeserver admins and users a better idea of what is happening with their account, who has access to it.

Note that the notes from Thunderbird are completely valid for email, but not for Matrix, as this 'many client, many homeservers' constraint means we have to have some sort of open client registration. Mastodon has something similar, Bluesky/ATProto as well… I'm not sure why it would not work for Matrix

sandhose · 2024-09-17T08:59:45Z

proposals/3861-delegated-oidc-architecture.md

+- When setting up a CAPTCHA challenge, CAPTCHA services expect the challenge to be served from a specific domain. Because the client can be on any domain, [Synapse currently advises disabling host verification](https://element-hq.github.io/synapse/latest/CAPTCHA_SETUP.html#getting-api-keys).
+  When this option is disabled, the CAPTCHA service expects the server to verify the domain of the challenge, which is not possible.
+- The agreement to the terms of service can be completely ignored by the client, as it only requires the client to send a request with no specific data to validate it.
+  This means the server has to trust the goodwill of the client to make sure the user agrees to the terms of service.
+- The current design of the email verification flow has a confusing user experience.
+  The link sent to the user by email leads to a screen where the user is prompted to return to their client.


This section talks about one particular example, which is also the homeserver where a lot of users will first experience this flow.

I'm not sure I understand the second part of your comment, how is this section talking about increasing the complexity onto all servers?

Tynach · 2024-09-18T19:09:23Z

The long-awaited big rewrite is here!

Shouldn't a big rewrite be a separate pull request/proposal? That way it's easier to distinguish between discussion about the old one, and discussion about the new one?

Edit: Though, I suppose doing that would require updating links everywhere that link to this proposal, and it would have potentially been less discoverable. Wish I'd have thought of that a few minutes earlier than I did.

marmarek · 2024-12-09T22:11:10Z

proposals/3861-delegated-oidc-architecture.md

+Another benefit of authenticating outside the client is that the client never has the user's full credentials.
+This has two important implications:
+
+- The client can't store the user's credentials, and thus can't use them to gain access without the user's consent.
+- The user can use different clients without worrying about revealing their account credentials to unknown parties. Only their homeserver ever interacts with their credentials.
+
+With the current authentication paradigm, sensitive account operations, such as changing passwords or deactivating the account, are protected with authentication steps where the client must present the user's credentials.
+
+With the current state of the ecosystem, in a password-based account, this means sending the user's password again: nothing prevents the client from storing the password on the initial login and using it to perform these actions.
+To put that in perspective, this means that if a user on the matrix.org homeserver tries to log in on a new client they want to try out, this client would be able to completely lock them out of their account by logging out other sessions and changing their password without additional confirmation from the user.
+
+This also effectively widens the attack surface for credential theft, as both the client and the homeserver currently have access to the user's credentials.
+
+Making it mandatory for the client to go through the system browser to authenticate means there is a part of the login flow that the client can't skip and doesn't have control over.
+The user has to give their explicit consent during that part of the flow, with no way for the client to bypass it.
+
+This opens up the possibility of giving more restrictive access to the user's account.
+It would open up Matrix to a new class of clients with only partial access to the user's account.
+
+**Note**: This does not change the level of trust that the user places in their homeserver. Their E2EE keys are still controlled by the client and never exposed to the homeserver, keeping the content of their events secret from the homeserver.


This whole point (and earlier to some degree too) feels like it got the trust levels backwards. In the E2EE scenario it's the client that is the trusted part, and the homeserver is less trusted. After all, it's the client who sees clear text messages, it's the client who can impersonate the user etc. This proposal does not change (nor should change!) any of this. And thus, saying that credentials are now isolated from the client doesn't matter much from the trust perspective - it can still access/decrypt your messages etc.

I realize there is some logic in this reasoning if you consider web client only - in that scenario indeed having user authentication on a separate homeserver-controlled page can isolate the client a bit from (potentially compromised!) homeserver by moving that workflow to another domain. And in the web scenario, also attaching credentials to the homeserver domain makes sense. But in the E2EE scenario, the web client is inherently less secure option anyway - the web client provider can stealthy insert any backdoor into your client and steal all the keys. With this change it may not be able to steal your account credentials anymore (at least as long as the user actually verifies the login is performed on a correct domain - which live shows users are really bad at...), but that's only a little consolation at this point.

But if you care more about your communication security, a much better choice is a local client (be it desktop, mobile etc), where the client code cannot change without your knowledge. In this scenario, requiring full blown web browser to open a homeserver-controlled page is a significant regression in terms of security. Now, not only you have extra attack surface (web browser with all its APIs - the fact that usually login page would be simple doesn't mean a compromised homeserver couldn't serve a login page using all kind of APIs), you now need to expose your credentials to an external application (so, you need to trust not only your local client, but also your web browser). And it gets even worse with containerized applications (flatpak, snap etc), where you either may need a full browser inside the container (wasteful) or you need to share your credentials with something outside of that container. And finally, in the native client case, the extra isolation that OIDC supposedly provide does not apply, as the client (in practice) controls what browser is used for providing credentials - so, if you have a malicious client, it can also give you fake browser that will intercept the credentials anyway.

So, I think at the very least this section needs adjustment to explain the impact on the non-web clients. But also, I would recommend to not only acknowledge it, but also mitigate it in the MSC - for example by still providing (not only in transitional period) a login workflow that doesn't require visiting an arbitrary website from your client (or other device). From a practical standpoint, it's only about logging into an account (and maybe invalidation of other sessions?). All other stuff like registration, deletion etc. can be handled via web interface - there is indeed very little point in requiring all of that to have complex and flexible UI in every client out there.

I'm not sure I'm following the trust model being backwards: in Matrix we always consider the client 'trusted' to decrypt/receive messages, but we also put things in place to limit that - the MSC doesn't read (to me) as conflicting with that. There's a note about not changing the E2EE model, which I think is here primarily to reduce confusion from the external audience.

What this MSC does change is the client's ability to make account-level changes on its own. UIA is something which can be driven almost entirely by code, and an infected client (either directly or indirectly) could trick the user into making their account inaccessible. For example, injecting code which says "please enter your password to confirm your account", which actually causes a password change in the background - for local clients, this could be injected by a virus on the user's machine, and for web clients it'd potentially be in the build pipeline. By having the authentication moved out of the client's implementation, users are significantly less likely to accidentally change their password without their knowledge.

Of course, the OIDC server could also be compromised and trick the user into changing their password, but the OIDC server will naturally have a higher sense of security given the threat vector.

For web client this separation makes some sense. I don't fully agree with the practical security benefit, mostly because I don't believe users indeed not giving password if the client asks for it, just because it's on the client's domain not a separate OIDC domain. And while this is more of a human issue than technical issue, I don't think it should be dismissed - it doesn't really matter from the user point of view how their account got compromised...
But for native clients (which normally provide much stronger E2EE properties, due to not trusting some remote system serving the client code) the situation gets significantly worse. Before, you needed to trust your client integrity (including OS running it) to protect your credentials and encryption keys. With this change, you still need to trust your client for credentials and encryption keys, but you also need to trust some random code fetched from the internet (from the client point of view) and executed on your system to not only handle your account credentials correctly (1), but also to not try to escape browser sandbox and steal your encryption keys (2).
You can argue that (1) is not much of a change since that "random code" is provided by the same service that will receive credentials anyway, but depending on the OIDC server implementation that may not be really true (and indeed it's common to serve static files, including js, separately, sometimes via 3rd-party CDNs). But (2) is a significant attack surface increase for the non-web client case.

I'm still struggling to follow the concern here, sorry. Using a web browser to "activate" a non-web browser client, even with a busted sandbox, should be no more risk than everyday use of the machine?

Ah, I see the missing step. Sometimes yes, sometimes no. You may have your client and/or your browser running in additional sandbox / container like Flatpak/Snap/AppImage - in which case you either need to start that browser for login in the same container, or punch some holes in the sandboxes. I believe similar concern applies to Android (and maybe other mobile?) where the client will need to add WebView that runs a code fetched from the OIDC server, in the same sandbox as the client itself.
But you can go further - in Qubes OS it's trivial to have a VM with just your client running and no browser at all. And there is an API for opening links in a different VM (can be even a disposable one), where by design such browser cannot communicate back to the VM with your client (otherwise that extra separation would be useless...). In this scenario, this MSC reduces benefits from this separation, as you'll need to allow running a browser in the same VM, at least for the login time.

I don't think the client needs to talk to the exact same web browser with OIDC to work. There are tons of headless and near-headless applications which poll the server to wait for credentials to appear, after authentication is done elsewhere. For example, logging in on your phone/TV might say "go to login.example.org to complete this login". A similar thing is achieved with the Vault command line if you have SSO set up: it'll poll the server until it gets credentials, having waited for the user to finish clicking buttons elsewhere.

Web clients, or clients in containers, can leverage this same step. The security model remains unchanged.

Okay, if this indeed will be implemented this way, then it should be fine, thanks for clarification.

Implementation is very much up to the server and client authors. At the protocol level, all we can do is make room for that to happen ;) (we are)

olivia-fl · 2024-12-11T03:46:10Z

proposals/3861-delegated-oidc-architecture.md

+This proposal introduces a new set of authentication APIs for Matrix, based on OAuth 2.0 and OpenID Connect (OIDC) specifications.
+
+As a first step, it introduces those APIs as altenratives to the existing User-Interactive Authentication (UIA) APIs, acknowledging the complexity of covering all the use cases of the existing APIs.
+The long-term goal is to deprecate the existing UIA APIs and replace them with the new OAuth 2.0/OIDC-based APIs.


An alternative that I think is worth discussing in the "alternatives" section is implementing OIDC while deprecating/removing all the existing auth types except m.login.password, m.login.dummy, and m.login.registration_token. The decision to remove m.login.password is very consequential, but the current text doesn't discuss the motivation, tradeoffs, or consequences of that decision.

(I want to be very explicit that I am talking about the decision to remove password auth, not the decision to implement OIDC or the decision to remove m.login.sso. I believe the latter two are well-covered by this MSC.)

If the intention is not to commit to remove existing UIA APIs here, then this sentence should possibly be removed or reworded.

This MSC does not remove anything from the spec. The SCT expects that the existing UIA flows won't go anywhere anytime soon, and that servers will follow suit - the proposal to use OIDC is to facilitate transition away from UIA, not replace it day 1.

As the SCT, we'll block attempts to remove UIA too quickly from the spec (and therefore implementations).

I guess the thing I want to understand is whether this MSC is a commitment to removing all existing UIA mechanisms at some point in the future, or whether it's just saying "we plan to do that, but might never do it if it turns out to be a bad decision". You say you'll block attempts to remove UIA too quickly, but is it also a possibility you will also block attempts to remove UIA at all or are you closing off that possibility by merging this MSC? If this statement isn't a commitment to do anything specific, then I don't care nearly as much.

How I read the current phrasing in the MSC is it's suggesting deprecation, which triggers a process for eventual consideration for removal. A different MSC would be required to both make the case for moving from deprecated to removal, and would be reviewed in that context (including counterarguments to say removal is inappropriate).

or in other words, any implied removal in this MSC is non-normative, meaning we're not bound to it.

Cool, I'm gonna not worry about it until then. Thanks for clarifying :)

olivia-fl · 2024-12-11T04:01:12Z

proposals/3861-delegated-oidc-architecture.md

+
+### OAuth 2.0 and OpenID Connect as building blocks
+
+Quoting the [specification](https://spec.matrix.org/latest/#introduction-to-the-matrix-apis):


The specification also currently states that:

The principles that Matrix attempts to follow are:
...

Keep It Simple & Stupid

provide a simple architecture with minimal third-party dependencies.

And that

The functionality that Matrix provides includes:
...

Managing user accounts (registration, login, logout)

OAuth2/OIDC is a complex third party dependency. Synapse+MAS is an substantial increase in architectural complexity over the current state. Bundling an OIDC server with non-synapse server implementations would be a similar increase in architectural complexity.

My understanding is that this MSC takes the position that managing user accounts should not be part of the functionality that matrix provides, and that it is worth introducing significant third party dependencies and architectural complexity to make that happen. It's inevitable that the documented principles will sometimes come into conflict, and changes may be made that violate some of them because the tradeoffs are worth it. If this is the intention, I would prefer that the decision to depart from the existing documented principles and core functionality was more explicit in the MSC text.

The dependency can be of varied complexity depending on what the implementation wants to achieve. Discussed in another thread, it's possible for a server implementation to expose a pretty small minimal interface and retain the same functionality as today's login flows, though servers like Synapse are intending to elect for higher complexity at the moment.

From a protocol design perspective, we're just not experts in authentication and would really like to outsource that to another protocol, taking on the dependency. Whether that dependency is OIDC or another authentication protocol is an open question though, and thoughts are welcome on that point. Currently, OIDC appears the most viable due to its widespread use, optional heavy complexity, and relative simplicity on the client side. There is different work for server developers to support it, but at its core it is capable of being added to a homeserver at the same complexity as today's UIA.

Ideally, there would be an implementation of that minimal interface somewhere to either prove it's possible, or prove I'm talking nonsense :p - this is not something the current team driving this proposal will be able to take on, but should hopefully be narrow-scoped enough for a third party implementation to happen, time permitting of course.

There is different work for server developers to support it, but at its core it is capable of being added to a homeserver at the same complexity as today's UIA.

That seems... rather unlikely to me. m.login.password UIA takes maybe 30m reading the spec to understand how to write an implementation. I've done a bit of server-side oauth2 work before and it was a lot more than 30m to learn how things work. I don't really have time or interest in trying to write a minimal implementation myself right now, and think I would be unlikely to persuade anybody by doing so (or by continuing to talk to spec people about it).

time permitting of course.

Again, unlikely I would attempt this, but out of curiosity what is the time requirement?

Mainly what I want is for this reasoning to end up in the actual MSC text, to benefit future readers that don't go reading through all the github comments.

proposals/3861-delegated-oidc-architecture.md

Matrix architecture change to delegate authentication via OIDC

4521510

hughns changed the title ~~MSCxxxx: Matrix architecture change to delegate authentication via OIDC~~ MSC3861: Matrix architecture change to delegate authentication via OIDC Aug 4, 2022

MSC3861

4550bbc

turt2live added proposal A matrix spec change proposal client-server Client-Server API kind:core MSC which is critical to the protocol's success needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Aug 4, 2022

ara4n added 3 commits August 9, 2022 11:29

typoe

00cd76d

typoes

b684b84

typoes

dd60163

MTRNord reviewed Aug 9, 2022

View reviewed changes

Add proposal for Matrix.org Foundation to become member of OpenID Fou…

9e58990

…ndation

turt2live reviewed Aug 11, 2022

View reviewed changes

proposals/3861-delegated-oidc-architecture.md Outdated Show resolved Hide resolved

proposals/3861-delegated-oidc-architecture.md Outdated Show resolved Hide resolved

ShadowJonathan mentioned this pull request Aug 14, 2022

Accomodate OIDC authentication ruma/ruma#1258

Open

greizgh reviewed Jan 1, 2023

View reviewed changes

proposals/3861-delegated-oidc-architecture.md Outdated Show resolved Hide resolved

hughns and others added 4 commits January 4, 2023 13:55

Update proposals/3861-delegated-oidc-architecture.md

fb0dfe7

Co-authored-by: greizgh <greizgh@ephax.org>

Move images inline

45c0b1c

Use term OpenID Provider

ba0c3dc

Add note about extending UIA as alternative

ef76bfa

MTRNord reviewed Feb 24, 2023

View reviewed changes

MTRNord mentioned this pull request Feb 24, 2023

MSC2966: Usage of OAuth 2.0 Dynamic Client Registration in Matrix #2966

Open

Add reference to related MSCs

4bffdc9

hughns mentioned this pull request Feb 28, 2023

MSC3970: Scope transaction IDs to devices #3970

Merged

turt2live added the matrix-2.0 Required for Matrix 2.0 label Mar 3, 2023

clokep mentioned this pull request May 12, 2023

Experimental MSC3861 support: delegate auth to an OIDC provider matrix-org/synapse#15582

Merged

5 tasks

clokep reviewed May 26, 2023

View reviewed changes

omentic mentioned this pull request Jun 20, 2023

Remove arbitrary restrictions on passwords cinnyapp/cinny#1285

Closed

devonh mentioned this pull request Jul 22, 2024

request: add 2FA element-hq/synapse#17459

Closed

Rework the MSC to better explain the rationale for the change

46adfad

sandhose changed the title ~~MSC3861: Matrix architecture change to delegate authentication via OIDC~~ MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC Sep 11, 2024

Start writing the actual proposal

ab746d5

sandhose added 2 commits September 16, 2024 15:45

Remove unused images

1f1ef22

Expand on 'why not just OIDC' and fix some typos

dc9d84b

Xiretza reviewed Sep 16, 2024

View reviewed changes

sandhose reviewed Sep 17, 2024

View reviewed changes

onestacked mentioned this pull request Sep 22, 2024

client,bridgev2: Add support for MSC4190 mautrix/go#288

Closed

Thatoo mentioned this pull request Nov 8, 2024

Testing YunoHost-Apps/conduwuit_ynh#1

Open

azmeuk mentioned this pull request Nov 13, 2024

SCIM provisioning API basic implementation element-hq/synapse#17144

Open

3 tasks

igogold mentioned this pull request Dec 4, 2024

Add a service of OpenID Provider (like Dex) to use with MAS spantaleev/matrix-docker-ansible-deploy#3852

Open

marmarek reviewed Dec 9, 2024

View reviewed changes

t3chguy mentioned this pull request Dec 10, 2024

MSC3266: Room summary API #3266

Open


		## Security considerations

		Please refer to individual proposals.


		### OAuth 2.0 and OpenID Connect as building blocks

		Quoting the [specification](https://spec.matrix.org/latest/#introduction-to-the-matrix-apis):

MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC #3861

Are you sure you want to change the base?

MSC3861: Next-generation auth for Matrix, based on OAuth 2.0/OIDC #3861

Conversation

hughns commented Aug 4, 2022 • edited by sandhose Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turt2live left a comment

Choose a reason for hiding this comment

MTRNord Feb 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hughns commented Feb 28, 2023

Choose a reason for hiding this comment

XxSNiPxX commented Jun 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Xiretza Sep 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandhose left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Tynach commented Sep 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

This comment was marked as outdated.

hughns commented Aug 4, 2022 •

edited by sandhose

Loading

MTRNord Feb 24, 2023 •

edited

Loading

Xiretza Sep 17, 2024 •

edited

Loading

Tynach commented Sep 18, 2024 •

edited

Loading