RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." #359

tyrelkostyk · 2018-07-12T17:27:45Z

I'm trying to validate Google's ID Tokens for user authentication on a web app. The id token can be decoded fine if I disable verification, but won't verify when I pass it the RSA256 Public Key. The Public Key in question is Base64urlUInt-Encoded (RFC 7518 Specification).

The Entire Public Key Response

  {
   "kty": "RSA",
   "alg": "RS256",
   "use": "sig",
   "kid": "8c9eb968f73744eaed421e48010142bce51a067f",
   "n": "uweJ3hFY9wqZ6ZG-iSNhQwHtKCGl8G_jcQgGPjOrS-Rum3dyDjicqkAyfS8XDn480KD_TZ5m-lqBjqfimePu2_cH4URDPIwsqSzJI2_piEhaqnXRptIe5YB5imAL6iETKaOPjw284Fc7EdHK-ekHMn3AXjsy9AIErwAVw4-4ZXXwHbyQXJy1DyUB4ZzxiEvw_qkQmLdltmrNkLOw-Xh-C9UkTZ9NA58bYPBnxLwnAu_ggw_g_-hCAs6OvXZbAfFHhIGBLyjtdDLVrfXo1112QREB9d5sEds0bKZtJcD9afl4E7Ht6G-g3jNP2clAu6-6B-cIe4-j8Ph1uJDPkAmDfw",
   "e": "AQAB"
  }

According to the Specification I linked above, the "n" parameter is the Modulus, and the "e" parameter is the exponent. I've tried absolutely every combination of decoding these to common Base64 format, but no matter what I do, pyJWT doesn't like it.

Expected Result

A Verified, decoded JSON data packet.

Actual Result

Traceback (most recent call last):
  File "/anaconda3/lib/python3.6/site-packages/jwt/algorithms.py", line 205, in prepare_key
    key = load_pem_private_key(key, password=None, backend=default_backend())
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py", line 20, in load_pem_private_key
    return backend.load_pem_private_key(data, password)
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1015, in load_pem_private_key
    password,
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1234, in _load_key
    self._handle_key_loading_error()
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1292, in _handle_key_loading_error
    raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jwt.py", line 93, in decode
    jwt, key=key, algorithms=algorithms, options=options, **kwargs
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jws.py", line 157, in decode
    key, algorithms)
  File "/anaconda3/lib/python3.6/site-packages/jwt/api_jws.py", line 221, in _verify_signature
    key = alg_obj.prepare_key(key)
  File "/anaconda3/lib/python3.6/site-packages/jwt/algorithms.py", line 207, in prepare_key
    key = load_pem_public_key(key, backend=default_backend())
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py", line 24, in load_pem_public_key
    return backend.load_pem_public_key(data)
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1041, in load_pem_public_key
    self._handle_key_loading_error()
  File "/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1292, in _handle_key_loading_error
    raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.

Reproduction Steps

import jwt

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

GoogPubKey = b'uweJ3hFY9wqZ6ZG-iSNhQwHtKCGl8G_jcQgGPjOrS-Rum3dyDjicqkAyfS8XDn480KD_TZ5m-lqBjqfimePu2_cH4URDPIwsqSzJI2_piEhaqnXRptIe5YB5imAL6iETKaOPjw284Fc7EdHK-ekHMn3AXjsy9AIErwAVw4-4ZXXwHbyQXJy1DyUB4ZzxiEvw_qkQmLdltmrNkLOw-Xh-C9UkTZ9NA58bYPBnxLwnAu_ggw_g_-hCAs6OvXZbAfFHhIGBLyjtdDLVrfXo1112QREB9d5sEds0bKZtJcD9afl4E7Ht6G-g3jNP2clAu6-6B-cIe4-j8Ph1uJDPkAmDfw'

#Convert to Base64 (replace '-', '_' with '+', '/', respectively, and pad with '=' to make multiple of 4)
GoogPubKey = GoogPubKey.replace(b'-', b'+')
GoogPubKey = GoogPubKey.replace(b'_', b'/')
GoogPubKey += b'=='
len(GoogPubKey) % 4  # 0
GoogPubKey = b'-----BEGIN PUBLIC KEY-----\n' + GoogPubKey + b'\n-----END PUBLIC KEY-----'

decoded = jwt.decode(IDjwt, GoogPubKey, algorithms='RS256')

Again, I've tried over a dozen different ways of using this RSA Public Key Google supplies, but nothing works (Using Base64url like they provide, using or not using the 'BEGIN PUBLIC KEY' prefix/suffix, type bytes or str, adding the "AQAB" prefix in multiple places, nothing works)

Any help would be greatly appreciated!! Thank you

System Information

$ python -m jwt.help

{
  "cryptography": {
    "version": "2.1.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.4"
  },
  "platform": {
    "release": "17.4.0",
    "system": "Darwin"
  },
  "pyjwt": {
    "version": "1.6.4"
  }
}

The text was updated successfully, but these errors were encountered:

thomasliub · 2018-07-19T07:08:25Z

The same issue.

jpadilla · 2018-07-19T13:33:39Z

Try something like this:

import jwt
from jwt.algorithms import RSAAlgorithm

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

key_json = '{"kty": "RSA","alg": "RS256","use": "sig","kid": "4129db2ea1860d2e871ee48506287fb05b04ca3f","n": "sxorUSxfZZjQL1mDr1rtbNGJE9lbVMiBmNZFqLhnQaefTfqMO3YgSlb_cptw5wS2Dn4phGNzjBaO1Hg5572mEqsmPl5z9MmybIOuqWXxYyIiCGWH3hoR2VPJ-1bN-SdszHb4ZWadXCCYqnHS216nrvHZK8vJyQ7XCchw43O00LC5Iwi2eKspQEj8YDQSZFsd7Mp2ULhKXVPyKeLH06aenBZZFwgjw8bow7MXS4uUkg4NOeH2iHNxclOYycg6Z87QrTVzHGBo9r-6s1XRTFh-rqcZC8RnR62wkPqB2AEHctOof_ZtaaDTZ1Xw7db8dRhhCnFkpiK_1d8c9N2Vm7Frxw","e": "AQAB"}'

public_key = RSAAlgorithm.from_jwk(key_json)

decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

0843C · 2018-08-08T19:42:58Z

try https://www.npmjs.com/package/jwk-to-pem

styk-tv · 2018-09-10T23:24:26Z

Hi. Arrived here after searching for the past 6 hours on the solution. My eyes are bleeding. My token is minted in Keycloak as asymmetric RS256.

Initial form retrieved from keycloak > realm > keys > rsa (and btw converted cert also matches public key), kid in the token matches kid in keycloak. its definitely the right key.

MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq33wpseciIME/pakwslqucEAC6f/T9lN1OYaNhFN3cs/50KhWuPu8918JZFECvtby835CIyIEngKWLFr+VPbe5GW94dujvlaZJOj0eGst3t2gd6TOeu5FwzsAJWHNP725fu5SwGlN2J81fmYSYAWG1QNK3Bu5Fn5KD0gCN1MRD8gjC+hXHte904fdwRxZdLfQinaEyW1xwlItsJ1U9/Ve6hZbE4HMZeyeGPrJna//xWbNi9xCize32L+pepyeXWGmcTgq7++p9bXu6xtm/8Pmt5KkuLS+sE1Lrj19sZffjeJoy5q6tTXr8CAJT5qU+P9km4WAdKkb+2IlWMmGtHyQwIDAQAB

Above key is raw, without headers. I found out they must be present for the key to be valid as the Crypto is making it as far as https://github.com/pyca/cryptography/blob/master/src/cryptography/hazmat/backends/openssl/backend.py#L1029-L1052 and then kills conversion with error producing

  File "/usr/local/lib/python3.6/site-packages/jwt/algorithms.py", line 207, in prepare_key
    key = load_pem_public_key(key, backend=default_backend())
  File "/usr/local/lib/python3.6/site-packages/cryptography/hazmat/primitives/serialization.py", line 24, in load_pem_public_key
    return backend.load_pem_public_key(data)
  File "/usr/local/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1040, in load_pem_public_key
    self._handle_key_loading_error()
  File "/usr/local/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1291, in _handle_key_loading_error
    raise ValueError("Could not deserialize key data.")
ValueError: Could not deserialize key data.

I found that headers are quite important for decoding as decisions as to what decoder to use is based on the header. -----BEGIN PUBLIC KEY----- vs -----BEGIN RSA PUBLIC KEY----- ref: https://tls.mbed.org/kb/cryptography/asn1-key-structures-in-der-and-pem and also new lines are important based on historical issue #257

So tried all combinations, exactly like @tyrelkostyk "no header, begin public, begin rsa public and no go.

For completion sending a token as well (to match against pasted key above)

eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJoSjZYWnF2cWVuVjVIay1qWnV5TUN5bGNPT3FnTzNpWlYtMnVBRTVhMDYwIn0.eyJqdGkiOiI4YjA3YzUyZi0wMWYzLTRjY2QtOTg2Yi04ODk2OWYzZTViZmIiLCJleHAiOjE1MzY2MTM2OTYsIm5iZiI6MCwiaWF0IjoxNTM2NjEzNjM2LCJpc3MiOiJodHRwczovL3NpZ25pbi5kZW1vLm1hZGNvcmUuY2xvdWQvYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoibG9jYWxob3N0Iiwic3ViIjoiMjUzNDk1ZmUtZGY2MC00NDE5LWExMzQtZWZhZTFiMGE5OWQwIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoibG9jYWxob3N0IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiYjg3Mzg5MDItYjY3ZS00MmE5LTlhZjAtYjU4ZTlhZDE1NWVjIiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsIm5hbWUiOiJQZXRlciBTdHlrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicG9sZmlsbSIsImdpdmVuX25hbWUiOiJQZXRlciIsImZhbWlseV9uYW1lIjoiU3R5ayIsImVtYWlsIjoicG9sZmlsbUBnbWFpbC5jb20ifQ.XPjYxLBlzo5tiaqEg3mwJwTpMaDzYleoeAUwc69ZrSqu-WtqIp6uSAPk_hsTnftTaCw9KSuCrvAi9csDjDswKST1rwUDpfSBOQ_lna_lxHMJFKgnVATpMmmoybyiAdn99kvISfrZ18nW8ECdBvOdJbPK-iI_WKq1NdxoilPPwOpsEAUT-hynK-astFoNQZn5mJegrAdU9LKho5D7Hs0A2I5i3nryyKJ2_Cak0HN8UB8BOBbiA_z0TWyeqY41VQFKqp-pNFMHltcQ3DeiSC1HK9bMjcy2AQ75ABy21xMlUtQJls_e000PgFClCpacayTCtj-udSHA5C23fRf3Qjsr5Q

Token is expired but the idea obviously to parse load public key instead before any other verifications. Two things: After this exercise i am certain I know far less than I thought I did. AND that this issue is most likely related to the formating of the public key. Decoding of token works fine obviously with verify=false

styk-tv · 2018-09-10T23:46:50Z

jwt.io validates the key with -----BEGIN RSA PUBLIC KEY-----/-----END RSA PUBLIC KEY----- header styles hence in my case corresponds to PKCS#1

and going in, selected line represents last working line, everything fails on that check,
you can see in debugger pub key made it all the way here.

This could still be a formatting issue as far as I suspect since online verifier at jwt.io was javascript based hence implemented differently.

styk-tv · 2018-09-11T01:22:53Z

@jpadilla Out of 1000's of messages i've read today, your hint WORKED.

@tyrelkostyk give up on passing public key yourself. Instead realy on OIDC well known config to discover exactly how you can construct RSA from JWK. It is there you don't have to guess it, at least it worked for me. Here is how I retrieved it.

Find your well-known config endpoint that lists all other endpoints. In case of keycloak its here (obviously realm will be different)

..then load "certs" endpoint

and then... use @jpadilla example above but with new token and detected jwk construction set

import jwt
from jwt.algorithms import RSAAlgorithm
IDjwt = 'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJoSjZYWnF2cWVuVjVIay1qWnV5TUN5bGNPT3FnTzNpWlYtMnVBRTVhMDYwIn0.eyJqdGkiOiI3M2Y0NDk2NC03M2UxLTRmZTgtYmRlZi1mNGJiY2FkNDQ5NjUiLCJleHAiOjE1MzY2MjY2NzksIm5iZiI6MCwiaWF0IjoxNTM2NjI2MDc5LCJpc3MiOiJodHRwczovL3NpZ25pbi5kZW1vLm1hZGNvcmUuY2xvdWQvYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoibG9jYWxob3N0Iiwic3ViIjoiMjUzNDk1ZmUtZGY2MC00NDE5LWExMzQtZWZhZTFiMGE5OWQwIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoibG9jYWxob3N0IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiMWZmYTgzZGItZDg3MS00YjQzLTk5YjgtOTMzNDQxNTg1OTExIiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsIm5hbWUiOiJQZXRlciBTdHlrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicG9sZmlsbSIsImdpdmVuX25hbWUiOiJQZXRlciIsImZhbWlseV9uYW1lIjoiU3R5ayIsImVtYWlsIjoicG9sZmlsbUBnbWFpbC5jb20ifQ.gaYPTNBLVYQJTRU5UHJ3GpVNKOrpekBIHmso6ZWLvaUJu4lgXf5wY1fdCqtsubTl2IlCy-zL81LABqua3He1M8qdAFvhKiTqyjm5SvzT40sjIDEnzCfUjYTvJIor8gcVscyNlqAuph0LNJb-aGu_tkaXjjLBl0DXqbyVyUtQ80ai9-ReCsmLobVLWeoyfi7hu9-elP6pOxaueIh5kT-MIux63xvvhSXBwQTxN9Dv7mNfSd1MCUUGtUJE1Fsb8dnIzpGjQh23Mw3p7SLq_ox_IhFP_mzRh3H8ye0gwVjILOOFUrCvFcylBS6TGhCZtkZP6luqRFbkCM9flQjQkON6lw'
key_json = '{"kid":"hJ6XZqvqenV5Hk-jZuyMCylcOOqgO3iZV-2uAE5a060","kty":"RSA","alg":"RS256","use":"sig","n":"q33wpseciIME_pakwslqucEAC6f_T9lN1OYaNhFN3cs_50KhWuPu8918JZFECvtby835CIyIEngKWLFr-VPbe5GW94dujvlaZJOj0eGst3t2gd6TOeu5FwzsAJWHNP725fu5SwGlN2J81fmYSYAWG1QNK3Bu5Fn5KD0gCN1MRD8gjC-hXHte904fdwRxZdLfQinaEyW1xwlItsJ1U9_Ve6hZbE4HMZeyeGPrJna__xWbNi9xCize32L-pepyeXWGmcTgq7--p9bXu6xtm_8Pmt5KkuLS-sE1Lrj19sZffjeJoy5q6tTXr8CAJT5qU-P9km4WAdKkb-2IlWMmGtHyQw","e":"AQAB"}'
public_key = RSAAlgorithm.from_jwk(key_json)
decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

TADA..

Thank you!!!

styk-tv · 2018-09-12T06:22:22Z

FYI:

Openidc well.known endpoint for Azure:
doc: https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-protocols-oidc
https://login.microsoftonline.com/{tenant}/v2.0/.well-known/openid-configuration

@tyrelkostyk and for Google:
doc: https://developers.google.com/identity/protocols/OpenIDConnect#discovery
https://accounts.google.com/.well-known/openid-configuration

sander-su · 2018-12-17T13:58:13Z

Just for reference, I'm also using keycloak and for me the issue was line breaks
(windows vs linux) and
adding linebreaks (after 76 characters) in the keystring (besides adding the header and footer)

vjsimha1 · 2018-12-28T23:05:43Z

Thank you @jpadilla! I was stuck for a long time but then came across this post and now I am able to successfully verify JWT token issued by AWS Cognito User Pool using the same guidelines.

Thanks once again!

donspaulding · 2019-01-22T23:42:01Z

@vjsimha1 Indeed, this is the solution I came up with for decoding AWS Cognito tokens:

import json
import jwt
import requests
from jwt.algorithms import RSAAlgorithm

def validate_cognito_token(id_token, cognito_region, cognito_user_pool_id, cognito_app_client_id):
    jwks = requests.get('https://cognito-idp.{aws_region}.amazonaws.com/{user_pool_id}/.well-known/jwks.json'.format(aws_region=cognito_region, user_pool_id=cognito_user_pool_id)).json()
    keys = {k['kid']: RSAAlgorithm.from_jwk(json.dumps(k)) for k in jwks['keys']}
    header = jwt.get_unverified_header(id_token)
    key_id = header['kid']
    algorithm = header['alg']
    pub_key = keys[key_id]
    # Next line raises errors if the audience isn't right or if the token is expired or has other errors.
    valid_token_data = jwt.decode(id_token, pub_key, audience=cognito_app_client_id, algorithm=algorithm)
    return valid_token_data

Does this look similar to your solution?

styk-tv · 2019-01-23T00:32:12Z

@donspaulding @vjsimha1

I documented my process in this PR vimalloc/flask-jwt-extended#222
Take a look at the doc, that setup works with Keycloak but should be very similar to Cognito. As a matter of fact we should start a catalog of all the public OIDC autoconfig url's in a gist somewhere.

There is no point of making the call to JWK endpoint and then retrieve all elements independently since they can be fed all at once into the RSAAlgorithm and then that public key is ready to go for verification of signature. Its a two lines of code if you use something like this: https://github.com/vimalloc/flask-jwt-extended/blob/master/flask_jwt_extended/tokens.py#L115

rudmac · 2019-02-01T14:55:30Z

same problem when trying to validate firebase tokens with the public key https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com

works only with verify=False

`
@app.post("/auth_firebase")
async def route_firebase_login_access_token(form_data: OAuth2FirebaseTokenRequestForm = Depends()):
token = form_data.token

header = jwt.get_unverified_header(token)

kid = header['kid']
alg = header['alg']
public_key = ""

with urllib.request.urlopen("https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com") as url:
    data = json.loads(url.read().decode())
    public_key = data[kid]

payload = jwt.decode(
    token, 
    algorithms=alg, 
    verify=False
)

# buggy:
#payload = jwt.decode(
#    token, 
#    public_key, 
#    algorithms=alg
#)

return {
    "payload": payload
}

`
the token test:
http://jwt.io

beaugogh · 2019-07-16T10:16:29Z

import jwt
from jwt.algorithms import RSAAlgorithm

IDjwt = <my id_token here>  # Decoding this with verify=False works correctly, so the problem isn't with the ID Token

key_json = '{"kty": "RSA","alg": "RS256","use": "sig","kid": "4129db2ea1860d2e871ee48506287fb05b04ca3f","n": "sxorUSxfZZjQL1mDr1rtbNGJE9lbVMiBmNZFqLhnQaefTfqMO3YgSlb_cptw5wS2Dn4phGNzjBaO1Hg5572mEqsmPl5z9MmybIOuqWXxYyIiCGWH3hoR2VPJ-1bN-SdszHb4ZWadXCCYqnHS216nrvHZK8vJyQ7XCchw43O00LC5Iwi2eKspQEj8YDQSZFsd7Mp2ULhKXVPyKeLH06aenBZZFwgjw8bow7MXS4uUkg4NOeH2iHNxclOYycg6Z87QrTVzHGBo9r-6s1XRTFh-rqcZC8RnR62wkPqB2AEHctOof_ZtaaDTZ1Xw7db8dRhhCnFkpiK_1d8c9N2Vm7Frxw","e": "AQAB"}'

public_key = RSAAlgorithm.from_jwk(key_json)

decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

just fyi, a simpler approach also worked for me;

import jwt
IDjwt = <my id_token here>
decoded = jwt.decode(IDjwt, '', verify=False)

rSrkn · 2019-09-17T06:15:45Z

Just for reference, I'm also using keycloak and for me the issue was line breaks
(windows vs linux) and
adding linebreaks (after 76 characters) in the keystring (besides adding the header and footer)

We also faced the same problem.
Our tests on mac had not problems but jenkins tests (slave docker) had 'Could not deserialize key data' error.
After applying above solution (adding line breaks ...) problem solved.

jordanmkoncz · 2019-12-19T01:54:46Z

Just a note that I faced a similar issue and found a solution. In my case, I was using a signing certificate downloaded from Auth0 in PEM format, which I had incorrectly assumed was a public key. According to https://cryptography.io/en/latest/hazmat/primitives/asymmetric/serialization/#pem:

A PEM block which starts with -----BEGIN CERTIFICATE----- is not a public or private key, it’s an X.509 Certificate. You can load it using load_pem_x509_certificate() and extract the public key with Certificate.public_key.

My PEM file did start with -----BEGIN CERTIFICATE-----, so I had to implement what is suggested by the cryptograpy library. Here's some sample code for doing this:

import jwt
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from django.conf import settings


def get_public_key_from_certificate(certificate_content: bytes) -> bytes:
    """
    Given a public signing certificate (e.g. downloaded from Auth0), load the certificate and then return the public key for it.

    :param certificate_content: The content of a public signing certificate in PEM format.
    :return: The public key.
    """
    certificate = x509.load_pem_x509_certificate(
        certificate_content, backend=default_backend()
    )

    return certificate.public_key()


def decode_access_token(access_token: str) -> dict:
    """
    Verify a JWT access token generated by Auth0, and return its decoded content.
    """
    public_key = get_public_key_from_certificate(settings.AUTH0_PUBLIC_CERTIFICATE)

    return jwt.decode(
        access_token,
        public_key,
        audience=settings.AUTH0_API_IDENTIFIER,
        algorithms=["RS256"],
        verify=True,
    )

Note that in this code, settings.AUTH0_PUBLIC_CERTIFICATE is a bytes value containing the content of the PEM key.

Now that I'm actually passing a public key into jwt.decode, everything works correctly.

See also: https://community.auth0.com/t/token-validation-with-python/21589/2.

EduardSchwarzkopf · 2020-08-04T05:13:17Z

@jpadilla Out of 1000's of messages i've read today, your hint WORKED.

@tyrelkostyk give up on passing public key yourself. Instead realy on OIDC well known config to discover exactly how you can construct RSA from JWK. It is there you don't have to guess it, at least it worked for me. Here is how I retrieved it.

Find your well-known config endpoint that lists all other endpoints. In case of keycloak its here (obviously realm will be different)

..then load "certs" endpoint

and then... use @jpadilla example above but with new token and detected jwk construction set

import jwt
from jwt.algorithms import RSAAlgorithm
IDjwt = 'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJoSjZYWnF2cWVuVjVIay1qWnV5TUN5bGNPT3FnTzNpWlYtMnVBRTVhMDYwIn0.eyJqdGkiOiI3M2Y0NDk2NC03M2UxLTRmZTgtYmRlZi1mNGJiY2FkNDQ5NjUiLCJleHAiOjE1MzY2MjY2NzksIm5iZiI6MCwiaWF0IjoxNTM2NjI2MDc5LCJpc3MiOiJodHRwczovL3NpZ25pbi5kZW1vLm1hZGNvcmUuY2xvdWQvYXV0aC9yZWFsbXMvbWFzdGVyIiwiYXVkIjoibG9jYWxob3N0Iiwic3ViIjoiMjUzNDk1ZmUtZGY2MC00NDE5LWExMzQtZWZhZTFiMGE5OWQwIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoibG9jYWxob3N0IiwiYXV0aF90aW1lIjowLCJzZXNzaW9uX3N0YXRlIjoiMWZmYTgzZGItZDg3MS00YjQzLTk5YjgtOTMzNDQxNTg1OTExIiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsIm5hbWUiOiJQZXRlciBTdHlrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicG9sZmlsbSIsImdpdmVuX25hbWUiOiJQZXRlciIsImZhbWlseV9uYW1lIjoiU3R5ayIsImVtYWlsIjoicG9sZmlsbUBnbWFpbC5jb20ifQ.gaYPTNBLVYQJTRU5UHJ3GpVNKOrpekBIHmso6ZWLvaUJu4lgXf5wY1fdCqtsubTl2IlCy-zL81LABqua3He1M8qdAFvhKiTqyjm5SvzT40sjIDEnzCfUjYTvJIor8gcVscyNlqAuph0LNJb-aGu_tkaXjjLBl0DXqbyVyUtQ80ai9-ReCsmLobVLWeoyfi7hu9-elP6pOxaueIh5kT-MIux63xvvhSXBwQTxN9Dv7mNfSd1MCUUGtUJE1Fsb8dnIzpGjQh23Mw3p7SLq_ox_IhFP_mzRh3H8ye0gwVjILOOFUrCvFcylBS6TGhCZtkZP6luqRFbkCM9flQjQkON6lw'
key_json = '{"kid":"hJ6XZqvqenV5Hk-jZuyMCylcOOqgO3iZV-2uAE5a060","kty":"RSA","alg":"RS256","use":"sig","n":"q33wpseciIME_pakwslqucEAC6f_T9lN1OYaNhFN3cs_50KhWuPu8918JZFECvtby835CIyIEngKWLFr-VPbe5GW94dujvlaZJOj0eGst3t2gd6TOeu5FwzsAJWHNP725fu5SwGlN2J81fmYSYAWG1QNK3Bu5Fn5KD0gCN1MRD8gjC-hXHte904fdwRxZdLfQinaEyW1xwlItsJ1U9_Ve6hZbE4HMZeyeGPrJna__xWbNi9xCize32L-pepyeXWGmcTgq7--p9bXu6xtm_8Pmt5KkuLS-sE1Lrj19sZffjeJoy5q6tTXr8CAJT5qU-P9km4WAdKkb-2IlWMmGtHyQw","e":"AQAB"}'
public_key = RSAAlgorithm.from_jwk(key_json)
decoded = jwt.decode(IDjwt, public_key, algorithms='RS256')

TADA..

Thank you!!!

This was exactly what i needed. i also struggled with keycloak. i had to make a small change to the jwt.decode though to make it working on my side:
decoded = jwt.decode(token, public_key, algorithms="RS256", audience="account")

Thanks alot for this solution!

vikramarsid · 2020-11-23T15:41:17Z

I could get this working without a public/private key using options.

decoded_details = jwt.decode(jwt_id_token, options={"verify_signature": False})

python -m jwt.help
{
  "cryptography": {
    "version": "3.2.1"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.5"
  },
  "platform": {
    "release": "19.6.0",
    "system": "Darwin"
  },
  "pyjwt": {
    "version": "2.0.0a1"
  }
}

Note: "verify_signature": False is not recommended for production use.

scottwn · 2021-07-22T02:22:01Z

Working with IBM Cloud tokens, @jpadilla's suggestion above cleared the could not deserialize error for me; however I'm now getting Expecting a PEM-formatted key:

>>> r = requests.get('https://iam.cloud.ibm.com/identity/keys')
>>> key = r.json()['keys'][1]
>>> public = jwt.algorithms.RSAAlgorithm.from_jwk(key)
>>> decoded = jwt.decode(token,key,algorithms=key['alg'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jwt.py", line 119, in decode
    decoded = self.decode_complete(jwt, key, algorithms, options, **kwargs)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jwt.py", line 90, in decode_complete
    decoded = api_jws.decode_complete(
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jws.py", line 149, in decode_complete
    self._verify_signature(signing_input, header, signature, key, algorithms)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jws.py", line 233, in _verify_signature
    key = alg_obj.prepare_key(key)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/algorithms.py", line 260, in prepare_key
    raise TypeError("Expecting a PEM-formatted key.")
TypeError: Expecting a PEM-formatted key.
>>> key = r.json()['keys'][2]
>>> public = jwt.algorithms.RSAAlgorithm.from_jwk(key)
>>> decoded = jwt.decode(token,key,algorithms=key['alg'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jwt.py", line 119, in decode
    decoded = self.decode_complete(jwt, key, algorithms, options, **kwargs)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jwt.py", line 90, in decode_complete
    decoded = api_jws.decode_complete(
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jws.py", line 149, in decode_complete
    self._verify_signature(signing_input, header, signature, key, algorithms)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/api_jws.py", line 233, in _verify_signature
    key = alg_obj.prepare_key(key)
  File "/home/scott/.pyenv/versions/3.9.5/lib/python3.9/site-packages/jwt/algorithms.py", line 260, in prepare_key
    raise TypeError("Expecting a PEM-formatted key.")
TypeError: Expecting a PEM-formatted key.

dajiaji · 2021-07-22T04:51:05Z

@scottwn At least, you should use public instead of key.

decoded = jwt.decode(token, public, algorithms=key['alg'])

tyrelkostyk changed the title ~~RSA256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data."~~ RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." Jul 12, 2018

styk-tv mentioned this issue Sep 12, 2018

ValueError: Could not deserialize key data. #347

Closed

bencever mentioned this issue Aug 19, 2019

GitHub app integration fails with jwt deserialization error getsentry/sentry#14434

Closed

3 tasks

alejandrodp mentioned this issue Jan 9, 2021

Token verification is not working, no error reported flavors/django-graphql-jwt#245

Open

amit-yp mentioned this issue Oct 10, 2021

Adds config to disable verify_signature of jwt matrix-org/synapse#11039

Closed

4 tasks

auvipy closed this as completed Nov 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." #359

RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." #359

tyrelkostyk commented Jul 12, 2018 •

edited

Loading

thomasliub commented Jul 19, 2018 •

edited

Loading

jpadilla commented Jul 19, 2018

0843C commented Aug 8, 2018 •

edited

Loading

styk-tv commented Sep 10, 2018

styk-tv commented Sep 10, 2018 •

edited

Loading

styk-tv commented Sep 11, 2018

styk-tv commented Sep 12, 2018

sander-su commented Dec 17, 2018

vjsimha1 commented Dec 28, 2018

donspaulding commented Jan 22, 2019

styk-tv commented Jan 23, 2019 •

edited

Loading

rudmac commented Feb 1, 2019 •

edited

Loading

beaugogh commented Jul 16, 2019 •

edited

Loading

rSrkn commented Sep 17, 2019

jordanmkoncz commented Dec 19, 2019 •

edited

Loading

EduardSchwarzkopf commented Aug 4, 2020

vikramarsid commented Nov 23, 2020

scottwn commented Jul 22, 2021

dajiaji commented Jul 22, 2021

RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." #359

RS256 Token Validation & Decoding using Public Key Not Working "ValueError: Could not deserialize key data." #359

Comments

tyrelkostyk commented Jul 12, 2018 • edited Loading

The Entire Public Key Response

Expected Result

Actual Result

Reproduction Steps

System Information

thomasliub commented Jul 19, 2018 • edited Loading

jpadilla commented Jul 19, 2018

0843C commented Aug 8, 2018 • edited Loading

styk-tv commented Sep 10, 2018

styk-tv commented Sep 10, 2018 • edited Loading

styk-tv commented Sep 11, 2018

styk-tv commented Sep 12, 2018

sander-su commented Dec 17, 2018

vjsimha1 commented Dec 28, 2018

donspaulding commented Jan 22, 2019

styk-tv commented Jan 23, 2019 • edited Loading

rudmac commented Feb 1, 2019 • edited Loading

beaugogh commented Jul 16, 2019 • edited Loading

rSrkn commented Sep 17, 2019

jordanmkoncz commented Dec 19, 2019 • edited Loading

EduardSchwarzkopf commented Aug 4, 2020

vikramarsid commented Nov 23, 2020

scottwn commented Jul 22, 2021

dajiaji commented Jul 22, 2021

tyrelkostyk commented Jul 12, 2018 •

edited

Loading

thomasliub commented Jul 19, 2018 •

edited

Loading

0843C commented Aug 8, 2018 •

edited

Loading

styk-tv commented Sep 10, 2018 •

edited

Loading

styk-tv commented Jan 23, 2019 •

edited

Loading

rudmac commented Feb 1, 2019 •

edited

Loading

beaugogh commented Jul 16, 2019 •

edited

Loading

jordanmkoncz commented Dec 19, 2019 •

edited

Loading