Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add OpenID Connect SSO support via django-allauth #1746

Closed
wants to merge 20 commits into from

Conversation

smkent
Copy link

@smkent smkent commented Oct 5, 2022

Proposed change

I use OpenID Connect (OIDC) for SSO with my self-hosted web apps. I've been excited to try out paperless-ngx for a while, but there is currently no support for OpenID Connect. This change adds support for OpenID Connect for SSO via django-allauth, with the potential for also supporting django-allauth's numerous SSO options.

OpenID Connect support enables SSO use via self-hosted providers like Authentik and Keycloak.

Note: This PR is based on my upstream PR in django-allauth. This code works with a small Dockerfile change (see below) but should be considered a working concept until my upstream PR is merged. django-allauth 0.52.0 has been released which contains support for OpenID Connect!

Sample OIDC configuration:

PAPERLESS_SSO_OIDC_NAME="Test OIDC"
PAPERLESS_SSO_OIDC_URL="https://some.oidc.server.example.com"
PAPERLESS_SSO_OIDC_CLIENT_ID="some-client-id-from-your-oidc-server"
PAPERLESS_SSO_OIDC_SECRET="accompanying-secret-from-your-oidc-server"

With no SSO configured, this PR does not change the login behavior:

paperless-password

However, SSO options will now appear on the login screen when configured:

paperless-both

If SSO is configured, it is also possible to hide the password form via a new setting PAPERLESS_LOGIN_HIDE_PASSWORD_FORM:

paperless-sso

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Other (please explain)

Checklist:

  • I have read & agree with the contributing guidelines.
  • If applicable, I have tested my code for new features & regressions on both ✅ mobile & ✅ desktop devices, using the latest version of major browsers.
  • If applicable, I have checked that all tests pass, see documentation.
  • I have run all pre-commit hooks, see documentation.
  • I have made corresponding changes to the documentation as needed.
  • I have checked my modifications for any breaking changes.

@smkent smkent requested a review from a team as a code owner October 5, 2022 05:37
@paperless-ngx-secretary paperless-ngx-secretary bot added backend non-trivial Requires approval by several team members labels Oct 5, 2022
@paperless-ngx-secretary
Copy link

Hello @smkent,

thank you very much for submitting this PR to us!

This is what will happen next:

  1. My robotic colleagues will check your changes to see if they break anything. You can see the progress below.
  2. Once that is finished, human contributors from paperless-ngx review your changes. Since this is a non-trivial change, a review from at least two contributors is required.
  3. Please improve anything that comes up during the review until your pull request gets approved.
  4. Your pull request will be merged into the dev branch. Changes there will be tested further.
  5. Eventually, changes from you and other contributors will be merged into main and a new release will be made.

Please allow up to 7 days for an initial review. We're all very excited about new pull requests but we only do this as a hobby.
If any action will be required by you, please reply within a month.

@stumpylog stumpylog added the work-in-progress Pull request which needs some changes before being able to merge label Oct 5, 2022
@coveralls
Copy link

coveralls commented Oct 5, 2022

Pull Request Test Coverage Report for Build 3909301335

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 60 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.9%) to 91.757%

Files with Coverage Reduction New Missed Lines %
paperless/settings.py 60 77.78%
Totals Coverage Status
Change from base Build 3896206563: -0.9%
Covered Lines: 5443
Relevant Lines: 5932

💛 - Coveralls

@shamoon
Copy link
Member

shamoon commented Oct 5, 2022

This is awesome, thanks for the PR. Let's keep an eye on your upstream PR and can circle back to this hopefully soon. When the time comes I think we should update the documentation to reflect this too. Bravo 👏

@shamoon shamoon marked this pull request as draft October 5, 2022 15:33
@shamoon shamoon added the blocked PR or issue that is not able to progress because of a dependency or other issue label Oct 12, 2022
@olant
Copy link

olant commented Nov 3, 2022

Looking forward for this PR being approved!

@ciphermenial
Copy link

Not sure if this will be useful at all but I did something similar a while ago and did a write up about it. https://ciphermenial.github.io/posts/adding-oauth-to-paperless/

@MartinVerges
Copy link

please merge! Using a central auth is an essential feature in my opinion

@stumpylog
Copy link
Member

This PR is a draft for the reasons stated when it opened. Once django-allauth has merged and released, it will get reviewed and merged.

@rmylb
Copy link

rmylb commented Dec 14, 2022

I'm waiting the merge because it's a really useful feature for my usecase !
Thank you for your work, it's amazing !

@lazyzyf
Copy link

lazyzyf commented Feb 9, 2023

There's also the pre-built image ghcr.io/smkent/paperless-ngx:feature-oidc-allauth

@DennisGaida That's detailed setup, thanks. I may be able to document setup on this side enough with those.

this image works perfect.

@codecov
Copy link

codecov bot commented Feb 9, 2023

Codecov Report

Merging #1746 (54542b1) into dev (40db244) will decrease coverage by 0.91%.
The diff coverage is 29.88%.

@@            Coverage Diff             @@
##              dev    #1746      +/-   ##
==========================================
- Coverage   93.11%   92.20%   -0.91%     
==========================================
  Files         140      142       +2     
  Lines        5997     6083      +86     
==========================================
+ Hits         5584     5609      +25     
- Misses        413      474      +61     
Flag Coverage Δ
backend 92.20% <29.88%> (-0.91%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/paperless/allauth_custom.py 0.00% <0.00%> (ø)
src/paperless/urls.py 100.00% <ø> (ø)
src/paperless/settings.py 77.63% <43.75%> (-6.42%) ⬇️
src/documents/templatetags/django_settings.py 83.33% <83.33%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@DennisGaida
Copy link
Contributor

DennisGaida commented Feb 10, 2023

One thing I have noticed:

  1. A new user is created for the OIDC user
  2. OIDC breaks internal authentication

I now have two users: "user" and "user8". user8 represents the OIDC user. the 8 seems to be some kind of autoincrement. When I delete the user in sqlite in auth_user as well as in socialaccount_socialaccount, and repeat the OIDC process I get a new user with a new "id", e.g. user7. With the OIDC changes enabled I can log in via OIDC - I can not login with the original user using regular login (wrong password).

Checking the DB logs (I added 'django.db.backends': { 'handlers': ['console'], 'level': 'DEBUG'}, in line 630 in settings.py), I see that my useraccount is queried and found, but somehow a new user is created. Maybe this is the default and is as it is supposed to be? A new user means new UI settings and views, and of course I only want one user.

I got around the duplicate user issue now by manually editing the socialaccount_socialaccount table, setting the socialaccount_socialaccount.user_id to my original user id (of "user"). I can now login via OIDC and I am the real user. I can also access the django admin panel like that. The other use that was created kind of is an orphan now and I just went ahead and deleted it... this of course shouldn't be the process and the socialaccount should just have been linked I suppose?

Once I disable the OIDC changes, I can login again with the original user.

Of course I'd love just one user and for both authentication methods to work at the same time.

Copy link
Member

@stumpylog stumpylog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably this is a misconfiguration on my part (but goes to show the need for good documentation on this feature), but I'm not able to get it working with Authelia. Attempting to log in using OIDC spins for sometime, before timing out.

@hendrik1120
Copy link

@stumpylog what kind of documentation do you have in mind?
For people with authelia and oidc already set up, it's just as easy as setting 4 variables and using paperless with https.
Only documentation for the variables would be needed and a warning about the redirect uri https issue.

Oidc in general really only makes sense to set up if you have a lot of services to log in to.
Providing a complete setup for keycloak or authelia with all the security considerations will be a long journey. These services also already have their own documentation available on how to integrate common apps, see https://www.authelia.com/integration/openid-connect/. Including paperless in there would also be an option.

@stumpylog
Copy link
Member

For people with authelia and oidc already set up, it's just as easy as setting 4 variables and using paperless with https.

Well, I already use Authelia and my own domain for https, but setting the 4 variables merely leads to timeouts. So I don't find it quite that easy and that's why I feel some additional documentation is still needed, or else there will be lots of reports to deal with. And clearly, I don't do web stuff often enough to deal with those.

@jonaswinkler
Copy link
Member

Hello!

First of all, I was able to get it working with Google OIDC. (I don't have Authelia avaiable, and don't really need it.)

Here are a couple issues I noticed:

  • Without digging further, authentication with normal users does not work on your branch (i.e., users created via python manage.py createsuperuser)
  • @shamoon merged the permissions feature into dev, and the user created at the end of the OIDC workflow lacks all permissions to do anything. I've already suggested internally that new users should probably get some minimum default permissions, but here they should probably get all admin permissions? Not sure. However, combine that with the issue above and I'm currently unable to do anything in paperless on this branch, since logging in with the other admin users doesn't work.
  • There are a lot of settings and code around PAPERLESS_SSO_MODULES and PAPERLESS_SSO_ENABLED. However, in your opening post you did not mention these at all. Are these two separate use cases? I'm not sure what I would need these for.

@jonaswinkler jonaswinkler marked this pull request as draft February 27, 2023 10:07
@raelix
Copy link

raelix commented Mar 1, 2023

this would be a great feature!

@mwllgr
Copy link

mwllgr commented Mar 5, 2023

Why is this PR marked as draft now?

@shamoon
Copy link
Member

shamoon commented Mar 5, 2023

Why is this PR marked as draft now?

Because it needs more work. We know folks are eager for the feature to be released, please limit comments to those that are useful to that goal. Thanks all.

@shamoon shamoon removed this from the v1.14 milestone Apr 2, 2023
@oblivioncth
Copy link

oblivioncth commented Apr 3, 2023

What DennisGaida noted indirectly touches on this, but something that doesn't seem to have been addressed yet is exactly how this interacts with existing users, and what OIDC claims are used when pulling the user from the provider.

A common flow is to use a user configurable claim (often 'prefered_username' or 'email') that is used when loading the user information from the provider. Then, if that field matches one from an existing user, that user is signed-in, if not, a new user is created with the info from the token or user info endpoint.

Alternatively, some implementations use the 'sub' claim as an ID to tie users in the local database to OIDC users and allow admins to change this value for any given user so that deployments can easily migrate their existing users over to OIDC. An unmatched subject means a new user.

The more flexible this is the better, but at the very least how it's currently handled should be documented.

@mwllgr

This comment was marked as off-topic.

@dkowis

This comment was marked as off-topic.

@XtremeOwnageDotCom

This comment was marked as off-topic.

@eldertek

This comment was marked as off-topic.

@AndrewNatoli
Copy link

Using Authentik

I see the discussion about working with existing accounts and getting more information on the configuration values, so here's my experience using Authentik.

Regarding linking existing accounts

In Authentik I created an oAuth2 provider. Under "Advanced protocol settings" there's an option called Subject Mode.

The options look like this:
image

When I originally set up PaperlessNgx with the base image, I registered with the same username I use for Authentik, hoping the Proxy auth example from PaperlessNg would still work-- it doesn't seem to, so thanks for making this feature!

Now then, I set up my provider to have its Subject Mode "Based on the User's username." This means if I was registered as cooluser on Authentik, it would tell Paperless my name is cooluser. It does do that in the socialaccount_socialaccount table under the uid column, but it created a new user in auth_user with the username of cooluser3 (which seems to be a random number instead of sequential).

I deleted the new user records from account_emailaddress, socialaccount_socialaccount, and auth_user (I believe that's the right order to avoid FK constraints for anyone who tries this themselves) then I switched the Subject mode to try email address.

The email address matched the one on my base Paperless account but still created a new user.

Ultimately I linked my Authentik login to my existing account by hand. I deleted the account_emailaddress and auth_user records for the new account, then went to the new record in socialaccount_socialaccount and revised the user_id to match the ID for my existing account.

I also tried with and without these environment variables which I may have interpreted wrong from the README:

      PAPERLESS_SSO_AUTO_LINK: FALSE
      PAPERLESS_SSO_AUTO_LINK_MULTIPLE: FALSE

My environment variables for Authentik

      PAPERLESS_SSO_OIDC_NAME: authentik
      PAPERLESS_SSO_OIDC_URL: "https://<my authentik server>/application/o/<provider slug>/"
      PAPERLESS_SSO_OIDC_CLIENT_ID: <client id from provider>
      PAPERLESS_SSO_OIDC_SECRET: <secret from provider>

The URL is the value of OpenID Configuration Issuer listed on the provider page.

Authentik Provider settings

  • OAuth2 Provider
  • Authorization flow-- I'm using default explicit
  • Client type is confidential
  • Redirect URL is https://<authentik host>/accounts/authentik/login/callback/

The authentik in my redirect URL matches the PAPERLESS_SSO_OIDC_NAME value.


Hope this helps with moving this feature forward!

@IamTaoChen
Copy link

IamTaoChen commented Jun 9, 2023

Maybe can sync group from OIDC. We know some IDP can offer group information.
For example, I use CAS and LDAP as back-end, I would send ldap-group information to OIDC at 'zoneinfo'.

like this (These code is writen for other project)

    def authenticate_user(self, user_info):
        if "email" in user_info:
            User = get_user_model()
            user, created = User.objects.get_or_create(email=user_info["email"])
            user = self.__user_id_sync(user,user_info)
            if created or settings.OIDC_USERINFO_SYNC:
                user = self.__user_info_sync(user,user_info)
            user.save()
            user.backend = "django.contrib.auth.backends.ModelBackend"
            return user
        messages.error(
            self.request,
            _(
                "Cannot get your email address. Did you try to"
                " log in with the correct account?"
            ),
        )
        return redirect("login_form")

    def __user_info_sync(self,user,user_info):
        if "name" in user_info:
            name = user_info["name"].split(" ")
            size = len(name)
            if size == 0:
                first_name = "user"
                second_name = ""
            elif len(name) == 1:
                first_name = name[0]
                second_name = ""
            else:
                first_name = name[0]
                second_name = name[1]
            user.first_name = first_name
            user.last_name = second_name
        user.save()
        return user

    def __user_id_sync(self,user, user_info):
        role = self.__check_role(user_info)
        user.role = role
        try:
            user.username = user_info["sub"]
        except:
            pass
        user.save()
        return user

    def __check_role(self, user_info):
        oidc_roles = self.__get_oidc_roles(user_info)
        if len(oidc_roles) == 0:
            return settings.OIDC_ROLE_DEFAULT
        return self.__analyze_role(oidc_roles)

    def __get_oidc_roles(self, user_info):
        tmp = user_info
        for path in settings.OIDC_ROLE_PATH_IN_RETURN:
            path = path.strip(",")
            if path == "":
                continue
            try:
                tmp = tmp[path]
            except KeyError:
                return []
        if isinstance(tmp, list):
            return tmp
        elif isinstance(tmp, str):
            return [tmp]
        else:
            return []

    def __analyze_role(self, oidc_roles):
        ROLE_ADMIN_NAME = "admin"
        ROLE_MANAGE_NAME = "manage"
        role_map = {
            ROLE_ADMIN_NAME: settings.OIDC_ROLE_ADMIN_PATTEREN,
            ROLE_MANAGE_NAME: settings.OIDC_ROLE_MANAGE_PATTEREN,
        }
        roles = []
        for key in role_map.keys():
            role_name = role_map[key]
            role_name = rf"{role_name}"
            if role_name.strip() == "":
                continue
            for oidc_role in oidc_roles:
                if re.search(role_name, oidc_role):
                    roles.append(key)
                    break
        if ROLE_ADMIN_NAME in roles:
            return 1
        elif ROLE_MANAGE_NAME in roles:
            return 2
        else:
            return settings.OIDC_ROLE_DEFAULT

@shamoon
Copy link
Member

shamoon commented Jun 18, 2023

@smkent its been many months with no activity so we’re going to close this. Again, I think others are interested in this feature so we hope to hear from you (or anyone else who is interested in moving this forward) in the future. Thank you for the contribution

@shamoon shamoon closed this Jun 18, 2023
@shamoon shamoon mentioned this pull request Jul 9, 2023
18 tasks
@github-actions
Copy link
Contributor

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion or issue for related concerns.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backend enhancement New feature non-trivial Requires approval by several team members notable Flag PRs to highlight in releases work-in-progress Pull request which needs some changes before being able to merge
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.