Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make stateproof verifier snark friendly #3895

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
ebd1016
change coin filps to be shake(sumhash(seed))
Feb 16, 2022
5446cae
add sequence of coin positions to the cert
Mar 10, 2022
4363c91
compute CC security using implied provenWeight
Mar 13, 2022
3ee4616
create a log2 appr func
Mar 15, 2022
6f4de56
fix stateproof message issues.
Mar 15, 2022
92ce531
remove proven weight form the coin hash
Mar 16, 2022
a21f7b7
fix cc test
Mar 17, 2022
f8168fe
use reject sampling in coin hash.
Mar 17, 2022
0afd674
use logarithmic approximation
Mar 27, 2022
b548344
refactoring
Mar 28, 2022
27c4e11
builder uses same appox function
Mar 31, 2022
b4c9be4
comments and doc
Mar 31, 2022
4ce22b6
handle negative value in number of reveals equation
Apr 4, 2022
40a9de2
add lnProvenWe to coinhash
Apr 10, 2022
2f275f3
fixed hash representation for coinhash
Apr 10, 2022
c76184b
fix CC benchmark
Apr 10, 2022
ac4df71
refactor
Apr 10, 2022
8911552
remove old numberofreveals code
Apr 10, 2022
593b62f
change secKQ to 256
Apr 11, 2022
e794180
fix CRs
Apr 12, 2022
c13ce05
CR fixes + rename
Apr 13, 2022
106e6f4
more CR fix
Apr 14, 2022
a3507ad
refactor the trusted params on the verifer.
Apr 14, 2022
3789b44
more refactoring
Apr 17, 2022
ec0543c
fix flaky test
Apr 17, 2022
6e35aad
remove Param structure
Apr 17, 2022
867d2cc
more fixes
Apr 17, 2022
cffa28a
update falcon lib + use byte as salt version
Apr 19, 2022
dd70755
add coinhash kat generator
Apr 20, 2022
485b44d
fix some CR comments
Apr 26, 2022
2cea300
clear out some documentation
Apr 27, 2022
2f03f65
Apply suggestions from code review
id-ms Apr 27, 2022
b566381
fix comments
Apr 28, 2022
ab2a2ff
refactor rejection sampling
Apr 28, 2022
112e5e7
CR fix
May 1, 2022
b7f38d9
refactoring
May 1, 2022
25a33d3
fix comments
May 3, 2022
30ee541
reduce the bytes allocated for stateproof message
May 3, 2022
ed9f141
Apply suggestions from code review
id-ms May 4, 2022
5ae415b
fix test since stateproof message hash was reduce
May 4, 2022
32a8164
fix CR comments
May 8, 2022
9c6a06a
more refactoring
May 8, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
refactor the trusted params on the verifer.
  • Loading branch information
algoidan committed Apr 17, 2022
commit a3507ada1c643c9dfc7e7a5d7bf027c92a4c817e
4 changes: 2 additions & 2 deletions compactcert/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,15 +64,15 @@ func (ccw *Worker) builderForRound(rnd basics.Round) (builder, error) {
}
ccw.Message = msg

p, err := ledger.CompactCertParams(msg, votersHdr, hdr)
p, err := ledger.CompactCertParams(votersHdr, hdr)
if err != nil {
return builder{}, err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning the error directly like this is problematic. When eventually the error is logged or reported to the user, it will be very difficult, if not impossible, to know where the error came from.
The convention followed is to wrap the received error before reporting it upwards.
For example, see below (which should be %w and not %v):

		ccw.log.Warnf("ccw.handleSigMessage(): decode: %v", err)

}

var res builder
res.votersHdr = votersHdr
res.voters = voters
res.Builder, err = compactcert.MkBuilder(p, voters.Participants, voters.Tree)
res.Builder, err = compactcert.MkBuilder(p, msg.IntoStateProofMessageHash(), uint64(hdr.Round), voters.Participants, voters.Tree)
if err != nil {
return builder{}, err
}
Expand Down
8 changes: 2 additions & 6 deletions compactcert/worker_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -295,9 +295,7 @@ func TestWorkerAllSigs(t *testing.T) {
require.False(t, overflowed)

ccparams := compactcert.Params{
Data: tx.Txn.CertMsg.IntoStateProofMessageHash(),
ProvenWeight: provenWeight,
Round: tx.Txn.CertIntervalLatestRound,
StrengthTarget: proto.CompactCertStrengthTarget,
}

Expand All @@ -307,7 +305,7 @@ func TestWorkerAllSigs(t *testing.T) {
verif, err := compactcert.MkVerifier(ccparams, voters.Tree.Root())
require.NoError(t, err)

err = verif.Verify(&tx.Txn.Cert)
err = verif.Verify(uint64(tx.Txn.CertIntervalLatestRound), tx.Txn.CertMsg.IntoStateProofMessageHash(), &tx.Txn.Cert)
require.NoError(t, err)
break
}
Expand Down Expand Up @@ -361,9 +359,7 @@ func TestWorkerPartialSigs(t *testing.T) {
require.False(t, overflowed)

ccparams := compactcert.Params{
Data: msg.IntoStateProofMessageHash(),
ProvenWeight: provenWeight,
Round: tx.Txn.CertIntervalLatestRound,
StrengthTarget: proto.CompactCertStrengthTarget,
}

Expand All @@ -372,7 +368,7 @@ func TestWorkerPartialSigs(t *testing.T) {

verif, err := compactcert.MkVerifier(ccparams, voters.Tree.Root())
require.NoError(t, err)
err = verif.Verify(&tx.Txn.Cert)
err = verif.Verify(uint64(tx.Txn.CertIntervalLatestRound), msg.IntoStateProofMessageHash(), &tx.Txn.Cert)
require.NoError(t, err)
}

Expand Down
3 changes: 1 addition & 2 deletions config/consensus.go
Original file line number Diff line number Diff line change
Expand Up @@ -382,8 +382,7 @@ type ConsensusParams struct {
// The threshold is computed as CompactCertWeightThreshold/(1<<32).
CompactCertWeightThreshold uint32

// CompactCertStrengthTarget is the security parameter for the compact
// certificate scheme.
// CompactCertStrengthTarget represents either k+q (for pre-quantum security) or k+2q (for post-quantum security)
algonautshant marked this conversation as resolved.
Show resolved Hide resolved
CompactCertStrengthTarget uint64

// EnableAssetCloseAmount adds an extra field to the ApplyData. The field contains the amount of the remaining
Expand Down
17 changes: 10 additions & 7 deletions crypto/compactcert/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ type sigslot struct {
// a compact certificate for that message.
type Builder struct {
Params
data StateProofMessageHash
round uint64
sigs []sigslot // Indexed by pos in participants
sigsHasValidL bool // The L values in sigs are consistent with weights
id-ms marked this conversation as resolved.
Show resolved Hide resolved
signedWeight uint64 // Total weight of signatures so far
Expand All @@ -57,16 +59,17 @@ type Builder struct {
// to be signed, as well as other security parameters, are specified in
// param. The participants that will sign the message are in part and
// parttree.
id-ms marked this conversation as resolved.
Show resolved Hide resolved
func MkBuilder(param Params, part []basics.Participant, parttree *merklearray.Tree) (*Builder, error) {
func MkBuilder(param Params, data StateProofMessageHash, round uint64, part []basics.Participant, parttree *merklearray.Tree) (*Builder, error) {
npart := len(part)
lnProvenWt, err := lnIntApproximation(param.ProvenWeight)
if err != nil {
return nil, err
}

b := &Builder{
Params: param,

Params: param,
data: data,
round: round,
sigs: make([]sigslot, npart),
sigsHasValidL: false,
signedWeight: 0,
Expand Down Expand Up @@ -104,9 +107,9 @@ func (b *Builder) IsValid(pos uint64, sig merklesignature.Signature, verifySig b
return err
}

cpy := make([]byte, len(b.Params.Data))
copy(cpy, b.Params.Data[:]) // TODO: onmce cfalcon is fixed can remove this copy.
if err := p.PK.VerifyBytes(uint64(b.Round), cpy, sig); err != nil {
cpy := make([]byte, len(b.data))
copy(cpy, b.data[:]) // TODO: onmce cfalcon is fixed can remove this copy.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this copy still needed here?
The TODO comment seems can be resolved.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still have a might have problem with c-falcon. when everything will be ready to merge into master we will remove it.

if err := p.PK.VerifyBytes(b.round, cpy, sig); err != nil {
id-ms marked this conversation as resolved.
Show resolved Hide resolved
return err
}
}
Expand Down Expand Up @@ -213,7 +216,7 @@ func (b *Builder) Build() (*Cert, error) {
lnProvenWeight: b.lnProvenWeight,
sigCommitment: c.SigCommit,
signedWeight: c.SignedWeight,
data: b.Params.Data,
data: b.data,
}

coinHash := makeCoinGenerator(&choice)
Expand Down
61 changes: 27 additions & 34 deletions crypto/compactcert/builder_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,7 @@ func generateTestSigner(firstValid uint64, lastValid uint64, interval uint64, a
return signer
}

func generateCertForTesting(a *require.Assertions) (*Cert, Params, crypto.GenericDigest, uint64) {
currentRound := basics.Round(compactCertRoundsForTests)
func generateCertForTesting(a *require.Assertions) (*Cert, Params, crypto.GenericDigest, uint64, StateProofMessageHash) {
// Doing a full test of 1M accounts takes too much CPU time in CI.
doLargeTest := false

Expand All @@ -91,10 +90,9 @@ func generateCertForTesting(a *require.Assertions) (*Cert, Params, crypto.Generi

npart := npartHi + npartLo

data := testMessage("hello world").IntoStateProofMessageHash()
param := Params{
Data: testMessage("hello world").IntoStateProofMessageHash(),
ProvenWeight: uint64(totalWeight / 2),
Round: currentRound,
StrengthTarget: compactCertStrengthTarget,
}

Expand All @@ -105,8 +103,8 @@ func generateCertForTesting(a *require.Assertions) (*Cert, Params, crypto.Generi
parts = append(parts, createParticipantSliceWithWeight(totalWeight, npartHi, key.GetVerifier())...)
parts = append(parts, createParticipantSliceWithWeight(totalWeight, npartLo, key.GetVerifier())...)

signerInRound := key.GetSigner(uint64(currentRound))
sig, err := signerInRound.SignBytes(param.Data[:])
signerInRound := key.GetSigner(compactCertRoundsForTests)
sig, err := signerInRound.SignBytes(data[:])
a.NoError(err, "failed to create keys")

for i := 0; i < npart; i++ {
Expand All @@ -116,7 +114,7 @@ func generateCertForTesting(a *require.Assertions) (*Cert, Params, crypto.Generi
partcom, err := merklearray.BuildVectorCommitmentTree(basics.ParticipantsArray(parts), crypto.HashFactory{HashType: HashType})
a.NoError(err)

b, err := MkBuilder(param, parts, partcom)
b, err := MkBuilder(param, data, compactCertRoundsForTests, parts, partcom)
a.NoError(err)

for i := 0; i < npart; i++ {
Expand All @@ -128,15 +126,15 @@ func generateCertForTesting(a *require.Assertions) (*Cert, Params, crypto.Generi
cert, err := b.Build()
a.NoError(err)

return cert, param, partcom.Root(), uint64(npart)
return cert, param, partcom.Root(), uint64(npart), data
}

func TestBuildVerify(t *testing.T) {
partitiontest.PartitionTest(t)

a := require.New(t)

cert, param, partCom, _ := generateCertForTesting(a)
cert, param, partCom, _, data := generateCertForTesting(a)

var someReveal Reveal
for _, rev := range cert.Reveals {
Expand All @@ -159,11 +157,11 @@ func TestBuildVerify(t *testing.T) {
verif, err := MkVerifier(param, partCom)
a.NoError(err)

err = verif.Verify(cert)
err = verif.Verify(compactCertRoundsForTests, data, cert)
a.NoError(err, "failed to verify the compact cert")
}

func generateRandomParticipant(a *require.Assertions, testname string) basics.Participant {
func generateRandomParticipant(a *require.Assertions) basics.Participant {
key := generateTestSigner(0, 8, 1, a)

p := basics.Participant{
Expand Down Expand Up @@ -205,10 +203,10 @@ func TestParticipationCommitmentBinaryFormat(t *testing.T) {
a := require.New(t)

var parts []basics.Participant
parts = append(parts, generateRandomParticipant(a, t.Name()))
parts = append(parts, generateRandomParticipant(a, t.Name()))
parts = append(parts, generateRandomParticipant(a, t.Name()))
parts = append(parts, generateRandomParticipant(a, t.Name()))
parts = append(parts, generateRandomParticipant(a))
parts = append(parts, generateRandomParticipant(a))
parts = append(parts, generateRandomParticipant(a))
parts = append(parts, generateRandomParticipant(a))

partcom, err := merklearray.BuildVectorCommitmentTree(basics.ParticipantsArray(parts), crypto.HashFactory{HashType: HashType})
a.NoError(err)
Expand All @@ -234,14 +232,12 @@ func TestSignatureCommitmentBinaryFormat(t *testing.T) {

a := require.New(t)

currentRound := basics.Round(compactCertRoundsForTests)
totalWeight := 10000000
numPart := 4

data := testMessage("test!").IntoStateProofMessageHash()
param := Params{
Data: testMessage("test!").IntoStateProofMessageHash(),
ProvenWeight: uint64(totalWeight / (2 * numPart)),
Round: currentRound,
StrengthTarget: compactCertStrengthTarget,
}

Expand All @@ -257,7 +253,7 @@ func TestSignatureCommitmentBinaryFormat(t *testing.T) {
}
parts = append(parts, part)

sig, err := key.GetSigner(uint64(currentRound)).SignBytes(param.Data[:])
sig, err := key.GetSigner(compactCertRoundsForTests).SignBytes(data[:])
require.NoError(t, err, "failed to create keys")
sigs = append(sigs, sig)

Expand All @@ -266,7 +262,7 @@ func TestSignatureCommitmentBinaryFormat(t *testing.T) {
partcom, err := merklearray.BuildVectorCommitmentTree(basics.ParticipantsArray(parts), crypto.HashFactory{HashType: HashType})
a.NoError(err)

b, err := MkBuilder(param, parts, partcom)
b, err := MkBuilder(param, data, compactCertRoundsForTests, parts, partcom)
a.NoError(err)

for i := 0; i < numPart; i++ {
id-ms marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -432,16 +428,14 @@ func TestBuilder_AddRejectsInvalidSigVersion(t *testing.T) {

// setting up a builder
a := require.New(t)
currentRound := basics.Round(compactCertRoundsForTests)

totalWeight := 10000000
npartHi := 1
npartLo := 9

data := testMessage("hello world").IntoStateProofMessageHash()
param := Params{
Data: testMessage("hello world").IntoStateProofMessageHash(),
ProvenWeight: uint64(totalWeight / 2),
Round: currentRound,
StrengthTarget: compactCertStrengthTarget,
}

Expand All @@ -453,12 +447,12 @@ func TestBuilder_AddRejectsInvalidSigVersion(t *testing.T) {
partcom, err := merklearray.BuildVectorCommitmentTree(basics.ParticipantsArray(parts), crypto.HashFactory{HashType: HashType})
a.NoError(err)

builder, err := MkBuilder(param, parts, partcom)
builder, err := MkBuilder(param, data, compactCertRoundsForTests, parts, partcom)
a.NoError(err)

// actual test:
signerInRound := key.GetSigner(uint64(currentRound))
sig, err := signerInRound.SignBytes(param.Data[:])
signerInRound := key.GetSigner(compactCertRoundsForTests)
sig, err := signerInRound.SignBytes(data[:])
require.NoError(t, err, "failed to create keys")
// Corrupting the version of the signature:
sig.Signature[1]++
Expand Down Expand Up @@ -492,8 +486,9 @@ func TestBuilderWithZeroProvenWeight(t *testing.T) {
a := require.New(t)

p := Params{ProvenWeight: 0}
data := testMessage("hello world").IntoStateProofMessageHash()

_, err := MkBuilder(p, nil, nil)
_, err := MkBuilder(p, data, compactCertRoundsForTests, nil, nil)
a.ErrorIs(err, ErrIllegalInputForLnApprox)

}
Expand All @@ -502,13 +497,11 @@ func BenchmarkBuildVerify(b *testing.B) {
totalWeight := 1000000
npart := 1000

currentRound := basics.Round(compactCertRoundsForTests)
a := require.New(b)

data := testMessage("hello world").IntoStateProofMessageHash()
param := Params{
Data: testMessage("hello world").IntoStateProofMessageHash(),
ProvenWeight: uint64(totalWeight / 2),
Round: compactCertRoundsForTests,
StrengthTarget: compactCertStrengthTarget,
}

Expand All @@ -522,8 +515,8 @@ func BenchmarkBuildVerify(b *testing.B) {
Weight: uint64(totalWeight / npart),
}

signerInRound := signer.GetSigner(uint64(currentRound))
sig, err := signerInRound.SignBytes(param.Data[:])
signerInRound := signer.GetSigner(compactCertRoundsForTests)
sig, err := signerInRound.SignBytes(data[:])
require.NoError(b, err, "failed to create keys")

partkeys = append(partkeys, signer)
Expand All @@ -539,7 +532,7 @@ func BenchmarkBuildVerify(b *testing.B) {

b.Run("AddBuild", func(b *testing.B) {
for i := 0; i < b.N; i++ {
builder, err := MkBuilder(param, parts, partcom)
builder, err := MkBuilder(param, data, compactCertRoundsForTests, parts, partcom)
if err != nil {
b.Error(err)
}
Expand All @@ -560,7 +553,7 @@ func BenchmarkBuildVerify(b *testing.B) {
b.Run("Verify", func(b *testing.B) {
for i := 0; i < b.N; i++ {
verif, _ := MkVerifier(param, partcom.Root())
if err = verif.Verify(cert); err != nil {
if err = verif.Verify(compactCertRoundsForTests, data, cert); err != nil {
b.Error(err)
}
}
Expand Down
2 changes: 1 addition & 1 deletion crypto/compactcert/coinGenerator.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ func makeCoinGenerator(choice *coinChoiceSeed) coinGenerator {
rep := crypto.HashRep(choice)
shk := sha3.NewShake256()
shk.Write(rep)

threshold, signedWt := prepareRejectionSamplingValues(choice.signedWeight)
return coinGenerator{shkContext: shk, signedWeight: signedWt, threshold: threshold}

Expand Down
2 changes: 1 addition & 1 deletion crypto/compactcert/committableSignatureSlot.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ func (cs *committableSignatureSlot) ToBeHashed() (protocol.HashID, []byte) {
}
binaryLValue := make([]byte, 8)
binary.LittleEndian.PutUint64(binaryLValue, cs.sigCommit.L)

sigSlotByteRepresentation := make([]byte, 0, len(binaryLValue)+len(cs.serializedSignature))
sigSlotByteRepresentation = append(sigSlotByteRepresentation, binaryLValue...)
sigSlotByteRepresentation = append(sigSlotByteRepresentation, cs.serializedSignature...)
Expand Down
15 changes: 6 additions & 9 deletions crypto/compactcert/structs.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,7 @@ const StateProofMessageHashType = crypto.Sha256

// Params defines common parameters for the verifier and builder.
type Params struct {
Data StateProofMessageHash
ProvenWeight uint64 // Weight proven by the certificate
Round basics.Round // The round for which the ephemeral key is committed to
ProvenWeight uint64 // Weight proven by the certificate
StrengthTarget uint64
}

Expand Down Expand Up @@ -64,12 +62,11 @@ type Reveal struct {
type Cert struct {
_struct struct{} `codec:",omitempty,omitemptyarray"`

SigCommit crypto.GenericDigest `codec:"c"`
SignedWeight uint64 `codec:"w"`
SigProofs merklearray.Proof `codec:"S"`
PartProofs merklearray.Proof `codec:"P"`
// CR salt version - chagne the falcon to byte or change this to int
MerkleSignatureSaltVersion int32 `codec:"v"`
SigCommit crypto.GenericDigest `codec:"c"`
SignedWeight uint64 `codec:"w"`
SigProofs merklearray.Proof `codec:"S"`
PartProofs merklearray.Proof `codec:"P"`
MerkleSignatureSaltVersion int32 `codec:"v"`
// Reveals is a sparse map from the position being revealed
// to the corresponding elements from the sigs and participants
// arrays.
Expand Down
Loading