Open
Description
The recently buf compatibility checks for KV protobufs interact badly withe the way we develop the 3 different Daml-LF protobufs: First a summary of the current state:
- The data in the Daml-LF protobufs is strictly versioned by the Daml-LF version. This is not the same as the SDK version, we can have multiple SDK releases without a new Daml-LF version.
- In addition to the stable Daml-LF versions, we have a 1.dev version. This one is explicitly for trying out experiments, no data continuity is provided, ledgers reject it by default so protobuf compatibility here is explicitly not a goal.
- However, the 1.dev protobufs needs to be wire compatible against older stable LF releases since we always use that for decoding. Enforcing anything more than wire-compatibility here is not useful.
- The daml-lf en/decoders have extensive hand-crafted backwards compatibility checks and we have integration tests as well for this.
- The KV protobufs reference currently both the language protobuf as well as values and transactions and they reference the 1.dev versions of those protobufs. The latter is deliberate, we want to be able to try out new features like exceptions on a KV stack.
- We disabled the check for the language protobuf in a4629a4 and LF: Move Archive and HashFunction out of daml-lf protobuf #11012 decouples things such that the 1.dev issue no longer arises.
So that leaves us with the buf checks for the transaction and value protobuf. Currently, those prevent us from developing the 1.dev versions in the way we want to because that could break compatibility with the 1.dev version of the latest stable SDK release.
My suggestion is the following:
- As part of the kv buf checks, exclude
com.daml.lf.transaction
andcom.daml.lf.value
. This gets us into a state where we can develop 1.dev as we want. Compatibility is not ensured via buf but via the existing tests for the en/decoder. - To recover the checks from buf as an extra safeguard, we keep a copy of the value & transaction protobuf for the latest stable Daml-LF version in the repository and we setup (wire-compat) buf checks for the 1.dev protobufs against the latest stable one.
Ideally we do 2 before 1 but given the extensive tests we have, if we end up being blocked on the current checks, we can do 1 before. The language team will start working on 2 with the estimate to finish it in the next month (from issue creation).
Activity