-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chanbackup: add new package implementing static channel backups #2370
chanbackup: add new package implementing static channel backups #2370
Conversation
652d509
to
1295005
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work on this package! Overall looks pretty complete to me, with healthy dose of unit tests :) No major comments on the approach at large. I do think moving to a TLV style format in the future though would be pretty useful, which fortunately is enabled by the versions on both single and multi backups!
Linter and vet ar both failing btw: https://travis-ci.org/lightningnetwork/lnd/jobs/472003458
Maybe just needs a rebase now that #2369 is in?
encryptionKey, err := genEncryptionKey(keyRing) | ||
if err != nil { | ||
return err | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
defer zeroing of encryption key?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we even really have any guarantees for stuff like that given the existence of the runtime and GC? If not, then it would just be security theatre.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can it hurt? it's at worst a marginal improvement, having one less copy in memory. it looks like Sum256
will copy a static array with the final result back to the caller, so would say its likely that there's at least one other reference
encryptionKey, err := genEncryptionKey(keyRing) | ||
if err != nil { | ||
return nil, err | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here wrt zeroing encryption key
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See above.
@Roasbeef needs rebase |
1295005
to
be84a5d
Compare
Comments responded to and rebased! PTAL @cfromknecht |
@cfromknecht ping! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Roasbeef another pass complete, almost ready. excellent coverage on this package with the extensive tests!
… baackups In this commit, we implement a series of new crypto operations that will allow us to encrypt and decrypt a set of serialized channel backups. Their various backups may have distinct encodings when serialized, but to the functions defined in this file, we treat them as simple opaque blobs. For encryption, we utilize chacha20poly1305 with a random 24 byte nonce. We use a larger nonce size as this can be safely generated via a CSPRNG without fear of frequency collisions between nonces generated. To encrypt a blob, we then use this nonce as the AD (associated data) and prepend the nonce to the front of the ciphertext package. For key generation, in order to ensure the user only needs their passphrase and the backup file, we utilize the existing keychain to derive a private key. In order to ensure that at we don't force any hardware signer to be aware of our crypto operations, we instead opt to utilize a public key that will be hashed to derive our private key. The assumption here is that this key will only be exposed to this software, and never derived as a public facing address.
In this commit, we add the initial implementation of the SCB structure. Given an SCB, and a user's seed, it will be possible to recover the settled balanced of a channel in the event of total or partial data loss. The SCB contains all information required to initiate the data loss protection protocol once we restore the channel and connect to the remote channel peer. The primary way outside callers will interact with this package are via the Pack and Unpack methods. Packing means writing a serialized+encrypted version of the SCB to an io.Writer. Unpacking does the opposite. The encoding format itself uses the same encoding as we do on the wire within Lightning. Each encoded backup begins with a version so we can easily add or modify the serialization format in the future, if new channel types appear, or we need to add/remove fields.
In this commit, we introduce the Multi sturct. Multi is a series of static channel backups. This type of backup can contains ALL the channel backup state in a single packed blob. This is suitable for storing on your file system, cloud storage, etc. Systems will be in place within lnd to ensure that one can easily obtain the latest version of the Multi for the node, and also that it will be kept up to date if channel state changes.
…nnels In this commit, we introduce a series of interfaces and methods that will allow external callers to backup either all channels, or a specific channel identified by its channel point. In order to abstract away the details w.r.t _how_ we obtain the set of open channels, or their storage mechanisms, we introduce a new LiveChannelSource interfaces. This interfaces allows us to fetch all channels, a channel by its channel point, and also all the known addresses for a node as we'll need this in order to connect out to the node in the case of a recovery attempt.
In this commit, we add a series of functions that will allow users to recover existing channel backups. We do this using two primary interfaces: the ChannelRestorer, and the PeerConnector. The first interfaces allows us to abstract away the details w.r.t exactly how a channel is restored. Instead, we simply expect that the channel backup will be inserted as a sort of "channel shell" which contains only the data required to initiate the data loss protection protocol. The second interface is how we instruct the Lightning node to connect out to the channel peer given its known addresses.
In this commit, we add a new MultiFile struct. We'll use this struct in store the latest multi-channel backup on disk, swap it out atomically, and finally extract+unpack the contents of the multi-file. The format that's written to disk is the same as a regular Packed multi. The contents of this new file are meant to be used to safely implement an always up to date multi file on disk as a way for users to easily rsync or fsnotiy (when it changes) the backup state of their channels. We implement an atomic update and swap in the UpdateAndSwap. The method uses relies on the underlying file system supporting an atomic rename syscall. We first make a temporary backup file, write the latest contents to that, then swap the temp file with the main file using rename(2). This way, we ensure that we always have a single up to date file, if the protocol aborts before the rename, then we can detect this, remove the temp file, and attempt another swap.
…p up to date In this commit, we introduce the chanbackup.SubSwapper interface. It takes a regular Swapper implementation (defined by the chanbackup.SubSwapper) interface along with a chanbackup.ChannelNotifier implementation. Given these two interfaces, we're able to be notified when a new channel is opened or closed, and then use the Swapper to atomically replace the on-disk channel back up. As a result, a Lightning daemon can ensure that they alwayts have a up to date channels.backup on disk that can safely be copied away by users and be used to restore channel funds in the event of partial/total data loss.
be84a5d
to
3b370fa
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM ⚾️itest flake seems unrelated, though will need a restart
switch s.Version { | ||
case DefaultSingleVersion: | ||
default: | ||
return fmt.Errorf("unable to de-serialize w/ unknown "+ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just making a note that once we add other versions, this should be deferred until after reading the length. then if we don't understand it, discard length
bytes and return to the caller so that we advance the reader. preferably this would be a concrete error so that the multi reader can skip/log it during deserialization
In this PR, we introduce a new package that implements our current scheme for static channel backups (SCB's). Note that the serialization format will likely change to use a TLV schema rather than a raw packed format. However the serialization is isolated to a single instance and can easily be modified when we decide on the final TLV format. As a result, this PR c an be reviewed as is.
Depends on #2369.
Static Channel Backup Scheme
Crypto
For encryption, we utilize
chacha20poly1305
with a random 24 byte nonce. We use a larger nonce size as this can be safely generated via a CSPRNG without fear of frequency collisions between nonces generated. To encrypt a blob, we then use this nonce as the AD (associated data) and prepend the nonce to the front of the ciphertext package.For key generation, in order to ensure the user only needs their passphrase and the backup file, we utilize the existing keychain to derive a private key. In order to ensure that at we don't force any hardware signer to be aware of our crypto operations, we instead opt to utilize a public key that will be hashed to derive our private key. The assumption here is that this key will only be exposed to this software, and never derived as a public facing address.
chanbackup.Single
The SCB contains all information required to initiate the data loss protection protocol once we restore the channel and connect to the remote channel peer.
The primary way outside callers will interact with this package are via the Pack and Unpack methods. Packing means writing a serialized+encrypted version of the SCB to an io.Writer. Unpacking does the opposite.
The encoding format itself uses the same encoding as we do on the wire within Lightning. Each encoded backup begins with a version so we can easily add or modify the serialization format in the future, if new channel types appear, or we need to add/remove fields. The backup contains:
chanPoint
of the channel.shortChanID
of the channel.keychain.KeyLocator
that allows us to re-derive the payment bas epoint we need to sweep our funds .keychain.KeyDescriptor
that we need in order to re-derive ourshachain
root to validate the information the remote party gives us during the DLP protocol. (see the next section for the complications that arose here)chanbackup.Multi
Multi is a series of static channel backups. This type of backup can contains ALL the channel
backup state in a single packed blob. This is suitable for storing on your file system, cloud storage, etc. Systems will be in place within lnd to ensure that one can easily obtain the latest version of the Multi for the node, and also that it will be kept up to date if channel state changes.
Broken off of #2313.