Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new smithy-modeled rules based endpoint resolution. #2214

Merged
merged 58 commits into from
Jul 31, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
fe4c3b3
Add aws standard lib functions (#2057)
isaiahvita Mar 20, 2023
c40b4dd
remove error collector for now (#2073)
isaiahvita Mar 29, 2023
f4381d4
add partition generator, and run gofmt (#2076)
isaiahvita Apr 5, 2023
f79579a
Add aws library functions for endpoint rules engine and fix partition…
isaiahvita May 5, 2023
0eb37e5
Add tests codegen and regen s3 (#2126)
isaiahvita May 17, 2023
72ed6c0
Add client config adapters (#2146)
isaiahvita Jun 12, 2023
98f1519
Add S3 Model Preprocessing for EP 2.0 (#2153)
Jun 21, 2023
50ceb26
Add BuiltIn generation (#2152)
isaiahvita Jun 23, 2023
7cb6d19
Add endpoint-based auth scheme resolution (#2158)
isaiahvita Jun 29, 2023
5a56b8a
Remove httpLabel trait from Bucket members (#2173)
isaiahvita Jul 7, 2023
b44fbbf
convert s3 handle_200_error_test into an actual unit test with a fake…
lucix-aws Jul 7, 2023
98caa35
Add functionality for non-SSL endpoints for backwards compatibility (…
isaiahvita Jul 7, 2023
f0fe836
fix borked go mod files
lucix-aws Jul 7, 2023
0839634
Fix Properties Auth Scheme Path (#2177)
Jul 7, 2023
260ce31
fix empty string signing name and region for sigv4 on ctx (#2178)
lucix-aws Jul 7, 2023
1229116
Check S3 signer customization insensitively (#2179)
Jul 7, 2023
151def9
Fix EP20 url encoding issues with S3 (#2181)
isaiahvita Jul 10, 2023
9d7de33
serde buckets into immutable hostname endpts (#2182)
lucix-aws Jul 11, 2023
774213d
adjust some expected errs
lucix-aws Jul 11, 2023
bd74619
remove incorrectly-applied {RequestRoute} filter
lucix-aws Jul 11, 2023
6921c80
fix borked serializers from last commit
lucix-aws Jul 11, 2023
50e8ce8
use httpbinding to escape instead
lucix-aws Jul 11, 2023
6c91a64
Check hostname immutable in EndpointDisableHttps and make it service …
isaiahvita Jul 11, 2023
4a484db
add back unintentionally deleted test cases (#2185)
isaiahvita Jul 11, 2023
85637ba
fix s3 go.mods
lucix-aws Jul 12, 2023
50d9d41
update expected errs for (2)
lucix-aws Jul 12, 2023
679e5d9
fix mrap error message asserts
lucix-aws Jul 12, 2023
233a39e
only serialize nonarn buckets on immutable host
lucix-aws Jul 12, 2023
97d4fb7
Delegate custom v1 resolvers to legacy customizations, remove adapter…
isaiahvita Jul 13, 2023
e2252e0
fix custom resolvers with path style (#2188)
isaiahvita Jul 13, 2023
3215621
add docs for migrating endpoint resolver v1
lucix-aws Jul 13, 2023
3fe6736
Fix S3Control endpoint customization tests (#2195)
isaiahvita Jul 19, 2023
1bbaab0
regen s3 and eventbridge; add context checker to eb; eb tests passing…
isaiahvita Jul 20, 2023
ecd96cc
Fix host prefix bug (#2200)
isaiahvita Jul 21, 2023
906b316
Fix broken S3 tests (#2203)
isaiahvita Jul 24, 2023
13216bc
don't serde the bucket (#2204)
lucix-aws Jul 25, 2023
09b3a36
ADD noserde to bucket instead of swapping with httplabel to prevent s…
lucix-aws Jul 25, 2023
5c4a4d3
mark EndpointResolverWithOptions deprecated
lucix-aws Jul 25, 2023
1a66c87
Added fallback behavior to custom v1 resolvers (#2206)
isaiahvita Jul 25, 2023
6872543
Make builtin resolver private (#2208)
isaiahvita Jul 25, 2023
5ce3521
remove unnecessary line thats breaking full build (#2209)
isaiahvita Jul 26, 2023
1dd1f49
fix no ruleset bug (#2210)
isaiahvita Jul 26, 2023
e4bb6df
regen for ep2
lucix-aws Jul 26, 2023
4fe37f7
suppress internal rulesfn lint
lucix-aws Jul 26, 2023
b330936
fix auth scheme test
lucix-aws Jul 26, 2023
9bbecdd
remove middleware dependencies on resolveendpointv2 (#2211)
isaiahvita Jul 27, 2023
8ac86b6
regen
lucix-aws Jul 27, 2023
3d0374d
regen sdk (#2213)
isaiahvita Jul 27, 2023
0ffe660
add changelog file
isaiahvita Jul 27, 2023
3b19a0d
temp patch ci workflow to use upstream smithy-go
lucix-aws Jul 27, 2023
9920f71
fix windows pathexp, point to ep20 smithygo for codegen
lucix-aws Jul 27, 2023
bbb3c26
have lint use ep2 smithy-go, netip -> net for min go version
lucix-aws Jul 27, 2023
87a6564
no source change: comment on ipv6 host id parse
lucix-aws Jul 27, 2023
99b20d3
regen sdk
isaiahvita Jul 27, 2023
3afdb0f
upgrade golint GH action
isaiahvita Jul 28, 2023
b092577
remove ep20 hardcoding in GH action
isaiahvita Jul 31, 2023
13394ca
fix deprecated comment
isaiahvita Jul 31, 2023
e9c9311
fixed changelog file
isaiahvita Jul 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add partition generator, and run gofmt (#2076)
  • Loading branch information
isaiahvita committed Jul 31, 2023
commit f4381d479f8fca554edbee20479aef5f37d4063f
8 changes: 4 additions & 4 deletions internal/endpoints/awsrulesfn/arn.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ type ARN struct {
Partition string
Service string
Region string
AccountId string
ResourceId OptionalStringSlice
AccountID string
ResourceID OptionalStringSlice
}

const (
Expand Down Expand Up @@ -54,8 +54,8 @@ func ParseARN(input string) *ARN {
Partition: sections[sectionPartition],
Service: sections[sectionService],
Region: sections[sectionRegion],
AccountId: sections[sectionAccountID],
ResourceId: splitResource(sections[sectionResource]),
AccountID: sections[sectionAccountID],
ResourceID: splitResource(sections[sectionResource]),
}
}

Expand Down
30 changes: 15 additions & 15 deletions internal/endpoints/awsrulesfn/arn_test.go
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
package awsrulesfn

import (
"testing"
"github.com/google/go-cmp/cmp"
"testing"
)

func TestParseARN(t *testing.T) {
cases := []struct {
input string
expect *ARN
input string
expect *ARN
}{
{
input: "invalid",
input: "invalid",
expect: nil,
},
{
input: "arn:nope",
input: "arn:nope",
expect: nil,
},
{
Expand All @@ -24,8 +24,8 @@ func TestParseARN(t *testing.T) {
Partition: "aws",
Service: "ecr",
Region: "us-west-2",
AccountId: "123456789012",
ResourceId: []string{"repository", "foo", "bar"},
AccountID: "123456789012",
ResourceID: []string{"repository", "foo", "bar"},
},
},
{
Expand All @@ -34,8 +34,8 @@ func TestParseARN(t *testing.T) {
Partition: "aws",
Service: "elasticbeanstalk",
Region: "us-east-1",
AccountId: "123456789012",
ResourceId: []string{"environment", "My App", "MyEnvironment"},
AccountID: "123456789012",
ResourceID: []string{"environment", "My App", "MyEnvironment"},
},
},
{
Expand All @@ -44,8 +44,8 @@ func TestParseARN(t *testing.T) {
Partition: "aws",
Service: "iam",
Region: "",
AccountId: "123456789012",
ResourceId: []string{"user", "David"},
AccountID: "123456789012",
ResourceID: []string{"user", "David"},
},
},
{
Expand All @@ -54,8 +54,8 @@ func TestParseARN(t *testing.T) {
Partition: "aws",
Service: "rds",
Region: "eu-west-1",
AccountId: "123456789012",
ResourceId: []string{"db", "mysql-db"},
AccountID: "123456789012",
ResourceID: []string{"db", "mysql-db"},
},
},
{
Expand All @@ -64,8 +64,8 @@ func TestParseARN(t *testing.T) {
Partition: "aws",
Service: "s3",
Region: "",
AccountId: "",
ResourceId: []string{"my_corporate_bucket", "exampleobject.png"},
AccountID: "",
ResourceID: []string{"my_corporate_bucket", "exampleobject.png"},
},
},
}
Expand Down
1 change: 0 additions & 1 deletion internal/endpoints/awsrulesfn/doc.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
// Package awsrulesfn provides AWS focused endpoint rule functions for
// evaluating endpoint resolution rules.

package awsrulesfn
7 changes: 7 additions & 0 deletions internal/endpoints/awsrulesfn/generate.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
//go:build codegen
// +build codegen

package awsrulesfn

//go:generate go run -tags codegen ./internal/partition/codegen.go -model partitions.json -output partitions.go
//go:generate gofmt -w -s .
3 changes: 1 addition & 2 deletions internal/endpoints/awsrulesfn/host.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
package awsrulesfn

import (
smithyhttp "github.com/aws/smithy-go/transport/http"
"net/netip"
"strings"
smithyhttp "github.com/aws/smithy-go/transport/http"

)

// IsVirtualHostableS3Bucket returns if the input is a DNS compatible bucket
Expand Down
16 changes: 8 additions & 8 deletions internal/endpoints/awsrulesfn/host_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ func TestIsVirtualHostableS3Bucket(t *testing.T) {
expect: true,
},
"single label no split too short": {
input: "a",
input: "a",
expect: false,
},
"single label with split": {
Expand All @@ -24,7 +24,7 @@ func TestIsVirtualHostableS3Bucket(t *testing.T) {
expect: true,
},
"multiple labels no split": {
input: "abc.123-",
input: "abc.123-",
expect: false,
},
"multiple labels with split": {
Expand All @@ -35,32 +35,32 @@ func TestIsVirtualHostableS3Bucket(t *testing.T) {
"multiple labels with split invalid label": {
input: "abc.123-...",
allowSubDomains: true,
expect: false,
expect: false,
},
"max length host label": {
input: "012345678901234567890123456789012345678901234567890123456789123",
expect: true,
},
"too large host label": {
input: "0123456789012345678901234567890123456789012345678901234567891234",
input: "0123456789012345678901234567890123456789012345678901234567891234",
expect: false,
},
"too small host label": {
input: "",
input: "",
expect: false,
},
"lower case only": {
input: "AbC",
input: "AbC",
expect: false,
},
"like IP address": {
input: "127.111.222.123",
input: "127.111.222.123",
expect: false,
},
"multiple labels like IP address": {
input: "127.111.222.123",
allowSubDomains: true,
expect: false,
expect: false,
},
}

Expand Down
114 changes: 114 additions & 0 deletions internal/endpoints/awsrulesfn/internal/partition/codegen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

73 changes: 73 additions & 0 deletions internal/endpoints/awsrulesfn/partition.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
package awsrulesfn

import "regexp"

// Partition provides the metadata describing an AWS partition.
type Partition struct {
ID string `json:"id"`
Regions map[string]PartitionConfig `json:"regions"`
RegionRegex string `json:"regionRegex"`
DefaultConfig PartitionConfig `json:"outputs"`
}

// PartitionConfig provides the endpoint metadata for an AWS region or partition.
type PartitionConfig struct {
Name *string `json:"name"`
DNSSuffix *string `json:"dnsSuffix"`
DualStackDNSSuffix *string `json:"dualStackDnsSuffix"`
SupportsFIPS *bool `json:"supportsFIPS"`
SupportsDualStack *bool `json:"supportsDualStack"`
}

const defaultPartition = "aws"

// GetPartition returns an AWS [Partition] for the region provided. If the
// partition cannot be determined nil will be returned.
func GetPartition(region string) *PartitionConfig {
return getPartition(partitions, region)
}

func getPartition(partitions []Partition, region string) *PartitionConfig {
for _, partition := range partitions {
if v, ok := partition.Regions[region]; ok {
v = mergePartition(v, partition.DefaultConfig)
return &v
}
}

for _, partition := range partitions {
regionRegex := regexp.MustCompile(partition.RegionRegex)
if regionRegex.MatchString(region) {
v := partition.DefaultConfig
return &v
}
}

for _, partition := range partitions {
if partition.ID == defaultPartition {
v := partition.DefaultConfig
return &v
}
}

return nil
}

func mergePartition(into PartitionConfig, from PartitionConfig) PartitionConfig {
if into.Name == nil {
into.Name = from.Name
}
if into.DNSSuffix == nil {
into.DNSSuffix = from.DNSSuffix
}
if into.DualStackDNSSuffix == nil {
into.DualStackDNSSuffix = from.DualStackDNSSuffix
}
if into.SupportsFIPS == nil {
into.SupportsFIPS = from.SupportsFIPS
}
if into.SupportsDualStack == nil {
into.SupportsDualStack = from.SupportsDualStack
}
return into
}
Loading