Skip to content

CDK CLI commands fail with "UnauthorizedException: Session token not found or invalid" after upgrading CDK to 2.173.0 #32510

Closed
@bbmarkus

Description

@bbmarkus

Please add your +1 👍 to let us know you have encountered this

Status: RESOLVED

Overview:

This failure happens under the following conditions:

  • CLI version 2.173.0
  • Credential source is SSO, configured in the /aws/config file.
  • A region is specified in the profile section of the config.
  • This region is different than the one in the sso_session, which is the one where the IAM Identity center was configured.

Complete Error Message:

Unable to determine the default AWS account (CredentialsProviderError): UnauthorizedException: Session token not found or invalid

Workaround:

Revert to version 2.172.0.

Solution:

In the call to the credential provider, pass the region in the parentClientConfig property instead of in the clientConfig property. The clientConfig is for configuring the inner auth client directly,
and has the highest priority, whereas parentClientConfig is the upper data client
and has lower priority than the sso_region but still higher priority than STS global region.

Related Issues:


Original issue

Describe the bug

Thiis morning after upgrading CDK installation to 2.173.0, all CDK command fail with the following error when using --verbose

Setting "CDK_DEFAULT_REGION" environment variable to eu-central-1
Unable to determine the default AWS account (CredentialsProviderError): UnauthorizedException: Session token not found or invalid

Rolling back to previous version 2.172.0 seems to fix this for us.

Regression Issue

  • Select this option if this issue appears to be a regression.

Last Known Working CDK Version

2.172.0

Expected Behavior

Command executes succefully

Current Behavior

All commands fail due to UnauthorizedException when trying to determine the default account.

Reproduction Steps

  1. Upgraded CDK and AWS cli (cli installed using the official snap package, upgrade was done as troubleshooting step)
  2. Using SSO for AWS cli login (default profile, if that can make a difference, i.e. not providing --profile as part of the commands)
  3. Run CDK command, e.g. cdk diff -e StackName --fail --verbose

Possible Solution

No response

Additional Information/Context

Command aws sts get-caller-identity returns the correct information, so this doesn't seem to be an authentication or SSO config issue.

CDK CLI Version

2.173.0

Framework Version

No response

Node.js Version

22

OS

Ubuntu (WSL)

Language

TypeScript

Language Version

No response

Other information

No response

Activity

added
bugThis issue is a bug.
needs-triageThis issue or PR still needs to be triaged.
on Dec 13, 2024
added
package/toolsRelated to AWS CDK Tools or CLI
potential-regressionMarking this issue as a potential regression to be checked by team member
on Dec 13, 2024
changed the title CDK CLI commands fail with UnauthorizedException after upgrading CDK to 2.172.0 CDK CLI commands fail with UnauthorizedException after upgrading CDK to 2.173.0 on Dec 13, 2024
ashishdhingra

ashishdhingra commented on Dec 13, 2024

@ashishdhingra
Contributor

@bbmarkus Good morning. Somehow using CDK version 2.173.0 (build b5c2189), I'm unable to reproduce the issue.

  • Setup SSO account in IAM Identity Center, assigning Permission Set to SSO user.
  • Setup SSO account using aws sso configure providing profile name as default. This configured SSO account in ~/.aws/config file as below:
    [default]
    sso_session = SSOAdmin
    sso_account_id = <<ACCOUNT-ID>>
    sso_role_name = AdministratorAccess
    region = us-west-2
    output = json
    
    [sso-session SSOAdmin]
    sso_start_url = https://d-<<SOME-ID>>.awsapps.com/start
    sso_region = us-west-2
    sso_registration_scopes = sso:account:access
    
  • Login to SSO using AWS CLI command aws sso login not providing profile name so that it uses default profile.
  • Using below CDK stack:
    import * as cdk from 'aws-cdk-lib';
    import { Bucket } from 'aws-cdk-lib/aws-s3';
    
    export class CdktestStackNew extends cdk.Stack {
      constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
        super(scope, id, props);
    
        new Bucket(this, 'testbucket');
      }
    }
    
    const app = new cdk.App();
    
    new CdktestStackNew(app, 'CdktestStackNew', {
      env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
    });
    Executed cdk diff -e CdktestStackNew --fail --verbose. It gives the below output:
    CDK toolkit version: 2.173.0 (build b5c2189)
    ...
    ...
    Reading cached notices from /Users/ashdhin/.cdk/cache/notices.json
    Toolkit stack: CDKToolkit
    Setting "CDK_DEFAULT_REGION" environment variable to us-west-2
    Looking up default account ID from STS
    Default account ID: <<ACCOUNT-ID>>
    Setting "CDK_DEFAULT_ACCOUNT" environment variable to <<ACCOUNT-ID>>
    ...  
    ...
    Retrieved account ID <<ACCOUNT-ID>> from disk cache
    Assuming role 'arn:aws:iam::<<ACCOUNT-ID>>:role/cdk-hnb659fds-lookup-role-<<ACCOUNT-ID>>-us-west-2'.
    Retrieved account ID <<ACCOUNT-ID>> from disk cache
    the stack 'CdktestStackNew' has not been deployed to CloudFormation or describeStacks call failed, skipping changeset 
    creation.
    Stack CdktestStackNew
    Parameters
    [+] Parameter BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically  retrieved from SSM Parameter Store. [cdk:skip]"}
    
    Conditions
    [+] Condition CDKMetadata/Condition CDKMetadataAvailable: {"Fn::Or":[{"Fn::Or":[{"Fn::Equals":[{"Ref":"AWS::Region"},"af-south-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-east-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-northeast-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-northeast-2"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-northeast-3"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-south-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-south-2"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-southeast-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-southeast-2"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-southeast-3"]}]},{"Fn::Or":[{"Fn::Equals":[{"Ref":"AWS::Region"},"ap-southeast-4"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ca-central-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"ca-west-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"cn-north-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"cn-northwest-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"eu-central-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"eu-central-2"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"eu-north-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"eu-south-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"eu-south-2"]}]},{"Fn::Or":[{"Fn::Equals":[{"Ref":"AWS::Region"},"eu-west-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"eu-west-2"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"eu-west-3"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"il-central-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"me-central-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"me-south-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"sa-east-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"us-east-1"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"us-east-2"]},{"Fn::Equals":[{"Ref":"AWS::Region"},"us-west-1"]}]},{"Fn::Equals":[{"Ref":"AWS::Region"},"us-west-2"]}]}
    
    Resources
    [+] AWS::S3::Bucket testbucket testbucketCEBFCF91
    
    
    ✨  Number of stacks with differences: 1
    Reading cached notices from /Users/ashdhin/.cdk/cache/notices.json
    

Executing aws sts get-caller-identity without profile name gives below output:

{
    "UserId": "<<SOME-ID>>:<<USERNAME>>",
    "Account": "<<ACCOUNT-ID>>",
    "Arn": "arn:aws:sts::<<ACCOUNT-ID>>:assumed-role/AWSReservedSSO_AdministratorAccess_<<SOME-ID>>/<<USERNAME>>"
}

Could you try executing aws sso login again? Unsure if your SSO session has expired.

Thanks,
Ashish

added
response-requestedWaiting on additional info and feedback. Will move to "closing-soon" in 7 days.
and removed
needs-triageThis issue or PR still needs to be triaged.
on Dec 13, 2024
wetherc

wetherc commented on Dec 13, 2024

@wetherc

Using a non-default profile, I'm able to reproduce the described error as follows:

  • AWS CLIV2 version: aws-cli/2.18.15 Python/3.12.6 Darwin/24.2.0 exe/x86_64
  • AWS CDK version: 2.173.0 (build b5c2189)

SSO configuraton:

[profile test-profile]
sso_session = mysession
sso_account_id = <Account ID>
sso_role_name = AdministratorAccess
region = us-east-2
output = json

[sso-session mysession]
sso_start_url = https://d-<some ID>.awsapps.com/start/#/
sso_region = us-east-1
sso_registration_scopes = sso:account:access

(Maybe notably, this is using a named profile; no default profile is configured)

  • Refresh SSO session with aws sso login --profile test-profile
  • Create a new test CDK stack:
#!/usr/bin/env python3

import os
import aws_cdk as cdk
from constructs import Construct


class TestStack(cdk.Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(self, construct_id, **kwargs)
        cdk.aws_s3.Bucket(scope, "testBucket")


app = cdk.App()
TestStack(
    app,
    "TestStack",
    env=cdk.Environment(
        account='<account ID>',
        region='us-east-2'
    ),
)

app.synth()
  • Attempt to diff stack with cdk diff --profile test-profile --verbose
  • Diff fails with
CDK toolkit version: 2.173.0 (build b5c2189)
Command line arguments: {
  _: [ 'diff' ],
  verbose: 1,
  v: 1,
  profile: 'test-profile',
  lookups: true,
...
...
Reading cached notices from /Users/<username>/.cdk/cache/notices.json
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to us-east-2
Unable to determine the default AWS account (CredentialsProviderError): UnauthorizedException: Session token not found or invalid
...
...
env: {
  CDK_DEFAULT_REGION: 'us-east-2',
  CDK_OUTDIR: 'cdk.out',
  CDK_CLI_ASM_VERSION: '38.0.1',
  CDK_CLI_VERSION: '2.173.0'
}
Need to perform AWS calls for account <account ID>, but no credentials have been configured
Reading cached notices from /Users/<username>/.cdk/cache/notices.json
Need to perform AWS calls for account <account ID>, but no credentials have been configured

Diffing the same stack with the same SSO configuration succeeds by forcibly downgrading to 2.172.0 (build 0f666c5)

aws sts get-caller-identity --profile test-profile returns

sts get-caller-identity
{
    "UserId": "<some ID>:<username>",
    "Account": "<account ID>",
    "Arn": "arn:aws:sts::<account ID>:assumed-role/AWSReservedSSO_AdministratorAccess_<some ID>/<username>"
}
removed
response-requestedWaiting on additional info and feedback. Will move to "closing-soon" in 7 days.
on Dec 13, 2024
ashishdhingra

ashishdhingra commented on Dec 13, 2024

@ashishdhingra
Contributor

Using a non-default profile, I'm able to reproduce the described error as follows:

  • AWS CLIV2 version: aws-cli/2.18.15 Python/3.12.6 Darwin/24.2.0 exe/x86_64
  • AWS CDK version: 2.173.0 (build b5c2189)

SSO configuraton:

[profile test-profile]
sso_session = mysession
sso_account_id = <Account ID>
sso_role_name = AdministratorAccess
region = us-east-2
output = json

[sso-session mysession]
sso_start_url = https://d-<some ID>.awsapps.com/start/#/
sso_region = us-east-1
sso_registration_scopes = sso:account:access

sts get-caller-identity
{
"UserId": ":",
"Account": "",
"Arn": "arn:aws:sts:::assumed-role/AWSReservedSSO_AdministratorAccess_/"
}

@wetherc Thanks for the inputs. I'm able to reproduce it by using different regions in sso-session and SSO profile:

...
Reading cached notices from /Users/ashdhin/.cdk/cache/notices.json
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to us-east-2
Unable to determine the default AWS account (CredentialsProviderError): UnauthorizedException: Session token not found or invalid
context: {
...
....
Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment
Reading cached notices from /Users/ashdhin/.cdk/cache/notices.json
Unable to resolve AWS account to use. It must be either configured when you define your CDK Stack, or through the environment

Below is my SSOAdmin profile configured in ~/.aws/config:

[sso-session SSOAdmin]
sso_start_url = https://d-<<SOME-ID>>.awsapps.com/start
sso_region = us-west-2
sso_registration_scopes = sso:account:access

[profile SSOAdmin]
sso_session = SSOAdmin
sso_account_id = <<ACCOUNT-ID>>
sso_role_name = AdministratorAccess
region = us-east-2
output = json

Ensured to execute aws sso logout --profile SSOAdmin and aws sso login --profile SSOAdmin to be sure.

Just checked, this works in CDK CLI (and package) version 2.172.0 (build 0f666c5).

18 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue is a bug.effort/mediumMedium work item – several days of effortp0package/toolsRelated to AWS CDK Tools or CLIpotential-regressionMarking this issue as a potential regression to be checked by team member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      CDK CLI commands fail with "UnauthorizedException: Session token not found or invalid" after upgrading CDK to 2.173.0 · Issue #32510 · aws/aws-cdk