Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crl provider: Static and FileWatcher provider implementations #6670

Merged
merged 63 commits into from
Oct 31, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
978cb44
rename certificateListExt to CRL
gtcooke94 Jun 29, 2023
7d032e0
CRLProvider file
gtcooke94 Jun 29, 2023
cdbc298
Add CRLProvider to RevocationConfig
gtcooke94 Jun 29, 2023
95991d8
Beginning refactor of CRL handling
gtcooke94 Jun 29, 2023
32e3158
Shell of StaticCRLProvider
gtcooke94 Jun 29, 2023
00de36e
basic static crl provider test
gtcooke94 Jun 29, 2023
8033cab
use loadCRL helper
gtcooke94 Jun 30, 2023
338a7f4
refactor of CRL loading
gtcooke94 Jun 30, 2023
d1f63fe
Table tests
gtcooke94 Jun 30, 2023
01afa97
Table tests
gtcooke94 Jun 30, 2023
401eb79
Add tests with Static CRL provider
gtcooke94 Jun 30, 2023
c88d12d
New certs to be used for CRL tests. Added test for passing and failin…
erm-g Sep 26, 2023
1feaae3
Main functionality of File Watcher (Directory) CRL provider
erm-g Sep 28, 2023
a9a84f1
Refactor async go routine, validate() func, add unit tests
erm-g Sep 29, 2023
5a0acad
Custom error callback, related unit tests
erm-g Sep 29, 2023
f3c830b
Error callback test improvement
erm-g Oct 1, 2023
4ea1b34
Comments for StaticCRLProvider
erm-g Oct 1, 2023
aeebd4e
Comments for public API
erm-g Oct 2, 2023
735ac20
go mod tidy
erm-g Oct 2, 2023
5c76a60
Comments for tests
erm-g Oct 2, 2023
0bc7757
Fix vet errors
erm-g Oct 2, 2023
f844c8c
Change Static provider behavior to match C Core, address other PR com…
erm-g Oct 3, 2023
a4da85e
Data race fix
erm-g Oct 4, 2023
c3ba07e
Test helper fn change
gtcooke94 Oct 4, 2023
ffe5c34
Address PR comments
erm-g Oct 8, 2023
6d28181
Address PR comments (part 2)
erm-g Oct 9, 2023
7814373
Migration from context to channel for controlling crl reloading gorou…
erm-g Oct 9, 2023
1a46b65
Align in-memory CRL updates during directory scan to C++ behavior
erm-g Oct 10, 2023
d7f1555
Improve comments for ScanCRLDirectory
erm-g Oct 10, 2023
2f1935d
Base test case for Scan CRL Directory file manipulations
erm-g Oct 10, 2023
0a7b086
full set of cases for CRL directory content manipulation
erm-g Oct 10, 2023
8d05f28
Add comment for table test structure
erm-g Oct 10, 2023
ccbf7f6
Fix for go.mod and go.sum
erm-g Oct 11, 2023
99ecab0
Empty directoru workaround
erm-g Oct 11, 2023
9e5a70d
Delete deprecated crl functionality
erm-g Oct 11, 2023
5643760
Restoring deprecated crl files
erm-g Oct 11, 2023
8898959
Fit to grpctest.Tester pattern
erm-g Oct 12, 2023
b16af8b
Update readme for crl provider tests
erm-g Oct 16, 2023
21f4301
Address PR comments
erm-g Oct 16, 2023
51b42aa
Revert "Restoring deprecated crl files"
gtcooke94 Oct 16, 2023
f0c1ca4
Merge remote-tracking branch 'upstream/master' into CrlTemp
gtcooke94 Oct 16, 2023
08188d1
Revert "Resolve conflicts with upstream - deletion of deprecated crl"
erm-g Oct 16, 2023
9b8d07e
Update link for gRFC proposal
erm-g Oct 19, 2023
ad15e23
Address PR comments
erm-g Oct 19, 2023
8e02546
Address PR comments part 1
erm-g Oct 20, 2023
e6a690d
Address PR comments part 2
erm-g Oct 20, 2023
340757d
Address PR comments part 3
erm-g Oct 20, 2023
1e4c5ac
Fix for go.mod and go.sum
erm-g Oct 20, 2023
f654d18
Fix comment typo
erm-g Oct 23, 2023
53d6b05
Fix for gRFC tag
erm-g Oct 23, 2023
1f398eb
Add more details to CRL api godoc comments.
erm-g Oct 23, 2023
f3dcca1
Address PR comments
erm-g Oct 24, 2023
131e6e7
Address PR comments
erm-g Oct 24, 2023
bc14ea8
Delete crl_deprecated.go and crl_deprecated_test.go
erm-g Oct 24, 2023
96bf905
Delete testdate/crl/provider/filewatcher directory and .gitignore und…
erm-g Oct 24, 2023
0ce6a2c
Race test fix
erm-g Oct 27, 2023
c57a08a
Address PR comments
erm-g Oct 27, 2023
4c53c56
Address PR comments
erm-g Oct 27, 2023
7fedab5
Refactor directory reloader test from checking size of crl map to que…
erm-g Oct 28, 2023
1025333
Add extra case for RefreshDuration config test
erm-g Oct 28, 2023
d9ba363
Update cpmment for table test structure
erm-g Oct 30, 2023
150e585
Unexport scan scanCRLDirectory, drop related mutex, update the comments
erm-g Oct 30, 2023
d7cf48f
Update API comments, clear tmp dir after the tests
erm-g Oct 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Change Static provider behavior to match C Core, address other PR com…
…ments
  • Loading branch information
erm-g committed Oct 3, 2023
commit f844c8cd889638d1388947780ac9786e65bf55e6
28 changes: 17 additions & 11 deletions security/advancedtls/advancedtls_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (
"errors"
"fmt"
"net"
"os"
"testing"

lru "github.com/hashicorp/golang-lru"
Expand Down Expand Up @@ -381,15 +382,20 @@ func (s) TestClientServerHandshake(t *testing.T) {
}

makeStaticCRLProvider := func(containsRevoked bool) *RevocationConfig {
cRLProvider := MakeStaticCRLProvider()
var crl *CRL

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: remove the blank line please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

rawCRLs := make([][]byte, 1)
var path string
if containsRevoked {
gtcooke94 marked this conversation as resolved.
Show resolved Hide resolved
crl = loadCRL(t, testdata.Path("crl/provider/crl_server_revoked.pem"))
path = testdata.Path("crl/provider/crl_server_revoked.pem")
} else {
crl = loadCRL(t, testdata.Path("crl/provider/crl_empty.pem"))
path = testdata.Path("crl/provider/crl_empty.pem")
}
cRLProvider.AddCRL(crl)

rawCRL, err := os.ReadFile(path)
if err != nil {
t.Fatalf("readFile(%v) failed err = %v", path, err)
}
rawCRLs = append(rawCRLs, rawCRL)
cRLProvider := MakeStaticCRLProvider(rawCRLs)
return &RevocationConfig{
AllowUndetermined: true,
CRLProvider: cRLProvider,
Expand Down Expand Up @@ -619,14 +625,14 @@ func (s) TestClientServerHandshake(t *testing.T) {
// server custom check fails
{
desc: "Client sets peer cert, reload root function with verifyFuncGood; Server sets bad custom check; mutualTLS",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you changed this test case and it's not a new one - just curious as to why you changed it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good catch, I was experimenting with some chains and accidentally submitted (it doesn't influence the outputs). Will revert it back

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

clientCert: []tls.Certificate{cs.ClientCert3},
clientGetRoot: getRootCAsForClientCRL,
clientCert: []tls.Certificate{cs.ClientCert1},
clientGetRoot: getRootCAsForClient,
clientVerifyFunc: clientVerifyFuncGood,
clientVType: CertVerification,
clientExpectHandshakeError: true,
serverMutualTLS: true,
serverCert: []tls.Certificate{cs.ServerCert3},
serverGetRoot: getRootCAsForServerCRL,
serverCert: []tls.Certificate{cs.ServerCert1},
serverGetRoot: getRootCAsForServer,
serverVerifyFunc: verifyFuncBad,
serverVType: CertVerification,
serverExpectError: true,
Expand Down Expand Up @@ -707,7 +713,7 @@ func (s) TestClientServerHandshake(t *testing.T) {
},
// Client: set valid credentials with the revocation config
// Server: set valid credentials with the revocation config
// Expected Behavior: success, because non of the certificate chains sent in the connection are revoked
// Expected Behavior: success, because none of the certificate chains sent in the connection are revoked
{
desc: "Client sets peer cert, reload root function with verifyFuncGood; Server sets peer cert, reload root function; mutualTLS",
clientCert: []tls.Certificate{cs.ClientCert1},
Expand Down
19 changes: 14 additions & 5 deletions security/advancedtls/crl_provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,24 +45,33 @@ type CRLProvider interface {
CRL(cert *x509.Certificate) (*CRL, error)
}

// StaticCRLProvider implements CRLProvider interface by accepting CRL structs
// and storing them in-memory.
// StaticCRLProvider implements CRLProvider interface by accepting raw content
// of CRL files at creation time and storing parsed CRL structs in-memory.
dfawley marked this conversation as resolved.
Show resolved Hide resolved
type StaticCRLProvider struct {
// TODO CRL is sort of our internal representation - provide an API for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the way this has been structured makes it easy for users to construct CRL objects?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// people to read into it, or provide a simpler type in the API then
// internally convert to this form
crls map[string]*CRL
}

// MakeStaticCRLProvider returns a new instance of the StaticCRLProvider.
func MakeStaticCRLProvider() *StaticCRLProvider {
// MakeStaticCRLProvider processes raw content of CRL files, adds parsed CRL
// structs into in-memory, and returns a new instance of the StaticCRLProvider.
func MakeStaticCRLProvider(rawCRLs [][]byte) *StaticCRLProvider {
dfawley marked this conversation as resolved.
Show resolved Hide resolved
p := StaticCRLProvider{}
p.crls = make(map[string]*CRL)
for idx, rawCRL := range rawCRLs {
cRL, err := NewCRL(rawCRL)
if err != nil {
grpclogLogger.Warningf("Can't parse raw CRL number %v from the slice: %v", idx, err)
continue
}
p.addCRL(cRL)
}
return &p
}

// AddCRL adds/updates provided CRL to in-memory storage.
func (p *StaticCRLProvider) AddCRL(crl *CRL) {
func (p *StaticCRLProvider) addCRL(crl *CRL) {
key := crl.CertList.Issuer.ToRDNSequence().String()
p.crls[key] = crl
}
Expand Down
11 changes: 8 additions & 3 deletions security/advancedtls/crl_provider_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ package advancedtls
import (
"crypto/x509"
"fmt"
"os"
"testing"
"time"

Expand All @@ -34,11 +35,15 @@ const nonCRLFilesUnderCRLDirectory = 5
// unrevoked, revoked leaf, and revoked intermediate chains, as well as a chain
// without CRL for issuer, and checks that it’s correctly processed.
func TestStaticCRLProvider(t *testing.T) {
p := MakeStaticCRLProvider()
rawCRLs := make([][]byte, 6)
for i := 1; i <= 6; i++ {
crl := loadCRL(t, testdata.Path(fmt.Sprintf("crl/%d.crl", i)))
p.AddCRL(crl)
rawCRL, err := os.ReadFile(testdata.Path(fmt.Sprintf("crl/%d.crl", i)))
if err != nil {
t.Fatalf("readFile(%v) failed err = %v", fmt.Sprintf("crl/%d.crl", i), err)
}
rawCRLs = append(rawCRLs, rawCRL)
}
p := MakeStaticCRLProvider(rawCRLs)
// Each test data entry contains a description of a certificate chain,
// certificate chain itself, and if CRL is not expected to be found.
tests := []struct {
Expand Down
10 changes: 7 additions & 3 deletions security/advancedtls/crl_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -509,11 +509,15 @@ func TestRevokedCert(t *testing.T) {
revokedLeafChain := makeChain(t, testdata.Path("crl/revokedLeaf.pem"))
validChain := makeChain(t, testdata.Path("crl/unrevoked.pem"))
cache, err := lru.New(5)
cRLProvider := MakeStaticCRLProvider()
rawCRLs := make([][]byte, 6)
for i := 1; i <= 6; i++ {
crl := loadCRL(t, testdata.Path(fmt.Sprintf("crl/%d.crl", i)))
cRLProvider.AddCRL(crl)
rawCRL, err := os.ReadFile(testdata.Path(fmt.Sprintf("crl/%d.crl", i)))
if err != nil {
t.Fatalf("readFile(%v) failed err = %v", fmt.Sprintf("crl/%d.crl", i), err)
}
rawCRLs = append(rawCRLs, rawCRL)
}
cRLProvider := MakeStaticCRLProvider(rawCRLs)
if err != nil {
t.Fatalf("lru.New: err = %v", err)
}
Expand Down
Loading