Add a guard when exercising by interface. #11836

sofiafaro-da · 2021-11-23T16:03:34Z

This fixes part of #11703, when exercising an inherited choice by
interface and you know the template id, via the command preprocessor.

It does this by inserting a "guard" in between the interface fetch and
the exercise body. The guard is a function Interface -> Bool, which
is general enough to check the template id, without complicating too
much in speedy. And can be generalized in the future to check more,
like signatories, etc.

I added the guard as an optional argument to UExerciseByInterface.
This isn't hooked up to the protobuf AST yet (or Haskell side for
that matter) -- but I'll do it in the next PR! For now you can invoke
the guarded exercise via the command preprocessor, so I can enable the
approprate engine tests. (There's still some failing fetch tests left,
but I decided to leave this for later. Fetch can be a lot simpler than
guarded choices, since you always add a fetch node. No need for fancy
continuations.)

changelog_begin
changelog_end

Pull Request Checklist

Read and understand the contribution guidelines
Include appropriate tests
Set a descriptive title and thorough description
Add a reference to the issue this PR will solve, if appropriate
Include changelog additions in one or more commit message bodies between the CHANGELOG_BEGIN and CHANGELOG_END tags
Normal production system change, include purpose of change in description
If you mean to change the status of a component, please make sure you keep the Component Status page up to date.

NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with /AzurePipelines run to
trigger the build.

cocreature

Nice, thank you!

daml-lf/engine/src/test/scala/com/digitalasset/daml/lf/engine/InterfacesTest.scala

daml-lf/language/src/main/scala/com/digitalasset/daml/lf/language/Ast.scala

cocreature · 2021-11-23T17:05:04Z

daml-lf/validation/src/main/scala/com/digitalasset/daml/lf/validation/Typing.scala

    ): Type = {
      checkExpr(cid, TContractId(TTyCon(tpl)))
      val choice = handleLookup(ctx, interface.lookupInterfaceChoice(tpl, chName))
      checkExpr(arg, choice.argBinder._2)
+      guard.foreach(guardExpr => checkExpr(guardExpr, TFun(TTyCon(tpl), TBool)))


we need to make sure we cover those in typechecking tests (doesn’t have to be in this PR)

added a TODO

remyhaemmerle-da

I would rather made the type of the guard Option (Interface -> Unit) instead of Option (Interface -> Bool).

We were thinking to do the same for precondition, (we did not because of backward compatibility)
We need to handle the case where the guard throw an exception (and insert transaction node). I believe it is already there.
It slightly simpler.
I do not see any reason why failing guard should not be catchable, in this case the ChoiceGuardFailed should be in a stable package like we have done ArithmeticError.

remyhaemmerle-da

After discussing with Moritz, I am now inclined in the total opposite direction:
I will let the type as it is : Option (Interface -> Boolean), however make failing guard non catchable. Indeed, we definitively do not want to insert a fetch node in case of failure, as we will not be able to replay the failing guard from the only information contains in the node.

One big caveat, is that we have to ensure that guards do not throw exception. So we have to catch any exception thrown by a guard and globally abort the transaction.

sofiafaro-da · 2021-11-24T10:42:21Z

After discussing with Moritz, I am now inclined in the total opposite direction: I will let the type as it is : Option (Interface -> Boolean), however make failing guard non catchable. Indeed, we definitively do not want to insert a fetch node in case of failure, as we will not be able to replay the failing guard from the only information contains in the node.

One big caveat, is that we have to ensure that guards do not throw exception. So we have to catch any exception thrown by a guard and globally abort the transaction.

Nice, ok. This PR should already have the right behavior then. I'll update the comments because they were talking about propagating exceptions.

This fixes part of #11703, when exercising an inherited choice by interface and you know the template id, via the command preprocessor. It does this by inserting a "guard" in between the interface fetch and the exercise body. The guard is a function Interface -> Bool, which is general enough to check the template id, without complicating too much in speedy. And can be generalized in the future to check more, like signatories, etc. I added the guard as an optional argument to UExerciseByInterface. This isn't hooked up to the protobuf AST yet (or Haskell side for that matter) -- but I'll do it in the next PR! For now you can invoke the guarded exercise via the command preprocessor, so I can enable the approprate engine tests. (There's still some failing fetch tests left, but I decided to leave this for later. Fetch can be a lot simpler than guarded choices, since you always add a fetch node. No need for fancy continuations.) changelog_begin changelog_end

When doing a "fetch by interface" command with a known template id, error out with a WronglyTypedContract if the fetched contract has a different template id. This doesn't affect daml, only affects replays, so it's rather minor. I also enabled the engine test that caught this. Part of #11703, follow up to #11836. changelog_begin changelog_end

…11862) * Prevent wrongly typed fetch by interface. When doing a "fetch by interface" command with a known template id, error out with a WronglyTypedContract if the fetched contract has a different template id. This doesn't affect daml, only affects replays, so it's rather minor. I also enabled the engine test that caught this. Part of #11703, follow up to #11836. changelog_begin changelog_end * strengthen test output checks

This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging. @SamirTalwar-DA is in charge of this release. Commit log: ``` 683ab87 Move ghc-lib{,-parser} to bazel-haskell-deps (#11775) 9350632 Fix releasing of resources in case connection initialization failed (#11915) e1559af Update `ModelConformanceValidator` comments and prevent them from getting outdated easily (#11924) 16a41f7 Avoid package validation in speedy compilation benchmark. (#11927) 16135e6 Limit supported input versions in damlc to >= LF 1.8 (#11905) 0ee4154 Use Absolute-indexes as keys for the Env-mapping during closure-conversion (#11912) 1d7bca8 Add optional typerep argument in UExerciseInterface. (#11910) c2c22f8 kvutils: Protos no longer depend on the Daml-LF transaction proto [KVL-1166] (#11909) 5641948 [Docs] Add labels to error codes to support references to them (#11913) 0e77676 Update protobuf docs template to handle oneOf (#11887) 5a9481f unify heading markup according to README.md (#11919) 61334cf kvutils - Add Writer which can handle deduplication periods as offsets [KVL-1172] (#11900) 0b9d57b Add ContractDoesntImplementInterface error. (#11884) 49e5d41 align index.rst files for HTML and PDF (#11907) dbbb05f Split daml-lf encode/decode Haskell libraries (#11906) e5d3902 iface: support for fixed choices in TS codegen (#11630) 31cc540 Turn package name & version warnings into an error (#11859) 4e50060 self-service compat: set branch name to not main (#11902) 2f4aa47 refactor to avoid impossible code path (#11901) a81995c switch dev images to Temurin (#11895) f3a0e2e Set scalafmt dialect explicitly (#11898) 60e372d Don't run pruning tests on H2, they time-out (#11897) 58e69ad LF: replace "dev" LF version by "1.dev" in bazel files (#11894) 8ef348d Use absolute stack locations in SExpr1 (#11877) 071bcf7 update NOTICES file (#11892) a1705d6 participant-state - Add an implicit logging context to the write service [kvl-1072] (#11838) 9ff64f7 Change daml script’s sleep to sleep for a minimum amount of time (#11886) 132c277 Add a Canton sandbox to the SDK (#11881) 68a2343 Only run self-service compat job on PRs (#11893) c27406c [DPP-762][Self-service error codes] Automate generation of inventory of error categories. #11879 1379722 Adapt the compatibility exclusions (#11872) d66ecc9 LF: Drop Archive Snapshot for LF < 1.14 (#11820) abc141b Increase pruning tests timeout (#11891) 66b4074 Update protobuf docs plugin (#11880) b0dda53 LF check stable proto with buf and md5sum. (#11888) 056fc52 Log while processing base64 encoded server key [DPP-761] (#11835) dbda67b bump JVM in Docker image (#11883) f69bd68 ledger-api-bench-tool: Fix flaky `MetricsCollectorSpec` (#11750) cb758e8 Fix call to experimental interface signatory builtin (#11882) 024400b Error when fetching the wrong template id (via fetch by interface). (#11862) 0852c8f Make DA.List.Total return Optional instead (#11878) df37346 [JSON-API] Add query store metrics (#11809) 2f8f69e Drop DA.Next.Set and DA.Next.Map (#11864) 5f3a4d2 [Self-service error codes] Fix section numbering in pdf for error codes section by moving it a level higher. (#11867) cf3ac01 [Self-service error codes] Do not return error code id and definite_answer in metadata for security sensitive errors (#11828) 026b92a Add gRPC definitions for participant user management service (#11818) 2fde30d Disable writing volatile bits in Scala statsfile (#11875) 4ed9ded Remove xxd from dev-env (#11876) eaded41 remove mergify (#11866) 3cd5028 fix a few more things in the daml-lf spec (#11851) beca0ee Refactor StandaloneApiServer factory (#11842) 6356f13 Properly upgrade gRPC to 1.41.0 (#11858) f6accd3 Release 1.18 RC2 (#11869) d858873 fix main (#11868) da8dd7e rotate release duty after 1.18.0-snapshot.20211123.8463.0.bd2a6852 (#11845) 066da4f [Self-service error codes] Small fixes for docs/scripts/live-preview.sh (#11856) 258fb65 Document how to deal with HTTP JSON API schema changes (#11336) b8937ad ci: self-service compat test start (#11853) de8d15f fix Nix install on macOS nodes (#11696) b3d1d40 Expose submissionId via the Java bindings (#11839) (#11847) 86da6e8 LF: Test scala interface type checking (#11833) 5f52f00 increase linux cluster size (#11860) 5c12d75 Add a guard when exercising by interface. (#11836) 7c3a2a7 Add a new KV submission failure error (#11854) aebc5a7 All packages must be valid (#11850) 0374843 speedy compilation benchmark (#11852) 393893a LF encoder: make package validation optional (#11849) 25b476f DPP-726 Add string interning unit tests (#11841) 59eb0d2 kvutils - For duplicate command rejections, add the submission id as metadata [KVL-1175] (#11848) 970243d Ensure stack-safety during closure-conversion. (#11778) e63c80d update LATEST (#11846) db42521 libs-scala: Change `SourceQueueResourceOwner` to `BoundedSourceQueueResourceOwner` [KVL-1177] (#11832) 109b606 Make the `InstrumentedSource.queue` use the `BoundedSourceQueue` [KVL-1177] (#11807) ``` Changelog: ``` - [Daml Compiler] The supported input LF versions for data-dependencies are now limited to LF 1.8 and newer. - [Daml2js] DARs with LF version < 1.8 are no longer supported. - [Integration Kit] kvutils protos no longer depend on the Daml-LF transaction proto - [Daml Standard Library] DA.List.Total functions now return Optional instead of being polymorphic in the return type. DA.Optional.Total has been removed. - [JSON-API] added metrics to separately track: - time taken to update query-store ACS (from ledger) - lookup times for the query store - [Daml Standard Library] DA.Next.Map and DA.Next.Set have been removed after being deprecated since Daml-LF 1.11 - [Ledger API] Introduce gRPC definitions for experimental user managament service to manage users and their rights for interacting with the Ledger API served by a participant node. [HTTP JSON API] [Docs] Document lack of data continuity guarantees and how to deal with schema changes [Java Bindings] submissionId is now exposed via the bindings, see issue #11705 [Integration Kit] Add a new SUBMISSION_FAILED internal error kvutils - For duplicate command rejections, the submission id of the already accepted transaction is returning as part of the gRPC metadata. The submission id will be included under the key `existing_submission_id`. - [Integration Kit] `SourceQueueResourceOwner` has been renamed to `BoundedSourceQueueResourceOwner` and takes a `BoundedSourceQueue` from now on - [Integration Kit] InstrumentedSource.queue.offer no longer returns a Future ``` CHANGELOG_BEGIN CHANGELOG_END

* release 2.0.0-snapshot.20211130.8536.0.683ab871 This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging. @SamirTalwar-DA is in charge of this release. Commit log: ``` 683ab87 Move ghc-lib{,-parser} to bazel-haskell-deps (#11775) 9350632 Fix releasing of resources in case connection initialization failed (#11915) e1559af Update `ModelConformanceValidator` comments and prevent them from getting outdated easily (#11924) 16a41f7 Avoid package validation in speedy compilation benchmark. (#11927) 16135e6 Limit supported input versions in damlc to >= LF 1.8 (#11905) 0ee4154 Use Absolute-indexes as keys for the Env-mapping during closure-conversion (#11912) 1d7bca8 Add optional typerep argument in UExerciseInterface. (#11910) c2c22f8 kvutils: Protos no longer depend on the Daml-LF transaction proto [KVL-1166] (#11909) 5641948 [Docs] Add labels to error codes to support references to them (#11913) 0e77676 Update protobuf docs template to handle oneOf (#11887) 5a9481f unify heading markup according to README.md (#11919) 61334cf kvutils - Add Writer which can handle deduplication periods as offsets [KVL-1172] (#11900) 0b9d57b Add ContractDoesntImplementInterface error. (#11884) 49e5d41 align index.rst files for HTML and PDF (#11907) dbbb05f Split daml-lf encode/decode Haskell libraries (#11906) e5d3902 iface: support for fixed choices in TS codegen (#11630) 31cc540 Turn package name & version warnings into an error (#11859) 4e50060 self-service compat: set branch name to not main (#11902) 2f4aa47 refactor to avoid impossible code path (#11901) a81995c switch dev images to Temurin (#11895) f3a0e2e Set scalafmt dialect explicitly (#11898) 60e372d Don't run pruning tests on H2, they time-out (#11897) 58e69ad LF: replace "dev" LF version by "1.dev" in bazel files (#11894) 8ef348d Use absolute stack locations in SExpr1 (#11877) 071bcf7 update NOTICES file (#11892) a1705d6 participant-state - Add an implicit logging context to the write service [kvl-1072] (#11838) 9ff64f7 Change daml script’s sleep to sleep for a minimum amount of time (#11886) 132c277 Add a Canton sandbox to the SDK (#11881) 68a2343 Only run self-service compat job on PRs (#11893) c27406c [DPP-762][Self-service error codes] Automate generation of inventory of error categories. #11879 1379722 Adapt the compatibility exclusions (#11872) d66ecc9 LF: Drop Archive Snapshot for LF < 1.14 (#11820) abc141b Increase pruning tests timeout (#11891) 66b4074 Update protobuf docs plugin (#11880) b0dda53 LF check stable proto with buf and md5sum. (#11888) 056fc52 Log while processing base64 encoded server key [DPP-761] (#11835) dbda67b bump JVM in Docker image (#11883) f69bd68 ledger-api-bench-tool: Fix flaky `MetricsCollectorSpec` (#11750) cb758e8 Fix call to experimental interface signatory builtin (#11882) 024400b Error when fetching the wrong template id (via fetch by interface). (#11862) 0852c8f Make DA.List.Total return Optional instead (#11878) df37346 [JSON-API] Add query store metrics (#11809) 2f8f69e Drop DA.Next.Set and DA.Next.Map (#11864) 5f3a4d2 [Self-service error codes] Fix section numbering in pdf for error codes section by moving it a level higher. (#11867) cf3ac01 [Self-service error codes] Do not return error code id and definite_answer in metadata for security sensitive errors (#11828) 026b92a Add gRPC definitions for participant user management service (#11818) 2fde30d Disable writing volatile bits in Scala statsfile (#11875) 4ed9ded Remove xxd from dev-env (#11876) eaded41 remove mergify (#11866) 3cd5028 fix a few more things in the daml-lf spec (#11851) beca0ee Refactor StandaloneApiServer factory (#11842) 6356f13 Properly upgrade gRPC to 1.41.0 (#11858) f6accd3 Release 1.18 RC2 (#11869) d858873 fix main (#11868) da8dd7e rotate release duty after 1.18.0-snapshot.20211123.8463.0.bd2a6852 (#11845) 066da4f [Self-service error codes] Small fixes for docs/scripts/live-preview.sh (#11856) 258fb65 Document how to deal with HTTP JSON API schema changes (#11336) b8937ad ci: self-service compat test start (#11853) de8d15f fix Nix install on macOS nodes (#11696) b3d1d40 Expose submissionId via the Java bindings (#11839) (#11847) 86da6e8 LF: Test scala interface type checking (#11833) 5f52f00 increase linux cluster size (#11860) 5c12d75 Add a guard when exercising by interface. (#11836) 7c3a2a7 Add a new KV submission failure error (#11854) aebc5a7 All packages must be valid (#11850) 0374843 speedy compilation benchmark (#11852) 393893a LF encoder: make package validation optional (#11849) 25b476f DPP-726 Add string interning unit tests (#11841) 59eb0d2 kvutils - For duplicate command rejections, add the submission id as metadata [KVL-1175] (#11848) 970243d Ensure stack-safety during closure-conversion. (#11778) e63c80d update LATEST (#11846) db42521 libs-scala: Change `SourceQueueResourceOwner` to `BoundedSourceQueueResourceOwner` [KVL-1177] (#11832) 109b606 Make the `InstrumentedSource.queue` use the `BoundedSourceQueue` [KVL-1177] (#11807) ``` Changelog: ``` - [Daml Compiler] The supported input LF versions for data-dependencies are now limited to LF 1.8 and newer. - [Daml2js] DARs with LF version < 1.8 are no longer supported. - [Integration Kit] kvutils protos no longer depend on the Daml-LF transaction proto - [Daml Standard Library] DA.List.Total functions now return Optional instead of being polymorphic in the return type. DA.Optional.Total has been removed. - [JSON-API] added metrics to separately track: - time taken to update query-store ACS (from ledger) - lookup times for the query store - [Daml Standard Library] DA.Next.Map and DA.Next.Set have been removed after being deprecated since Daml-LF 1.11 - [Ledger API] Introduce gRPC definitions for experimental user managament service to manage users and their rights for interacting with the Ledger API served by a participant node. [HTTP JSON API] [Docs] Document lack of data continuity guarantees and how to deal with schema changes [Java Bindings] submissionId is now exposed via the bindings, see issue #11705 [Integration Kit] Add a new SUBMISSION_FAILED internal error kvutils - For duplicate command rejections, the submission id of the already accepted transaction is returning as part of the gRPC metadata. The submission id will be included under the key `existing_submission_id`. - [Integration Kit] `SourceQueueResourceOwner` has been renamed to `BoundedSourceQueueResourceOwner` and takes a `BoundedSourceQueue` from now on - [Integration Kit] InstrumentedSource.queue.offer no longer returns a Future ``` CHANGELOG_BEGIN CHANGELOG_END * bump to include fix for damlc package validation changelog_begin changelog_end Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

sofiafaro-da requested review from cocreature and remyhaemmerle-da as code owners November 23, 2021 16:03

sofiafaro-da requested a review from nickchapman-da November 23, 2021 16:03

cocreature approved these changes Nov 23, 2021

View reviewed changes

sofiafaro-da requested a review from a team as a code owner November 23, 2021 17:35

remyhaemmerle-da reviewed Nov 24, 2021

View reviewed changes

sofiafaro-da added 6 commits November 24, 2021 11:11

scalafmt

5f793f3

Feedback and fix matches

18f4897

Update comments, we are always going to abort the transaction

a6a4a99

Raise WronglyTypedContract in SBGuardTemplateId.

5c2af3b

rebase and fix parser

aeb7f8a

sofiafaro-da force-pushed the interface-exercise-guard branch from 073aa2a to aeb7f8a Compare November 24, 2021 11:25

sofiafaro-da added 2 commits November 24, 2021 11:50

restore ANF

13d171c

scalafmt

b476566

sofiafaro-da enabled auto-merge (squash) November 24, 2021 12:28

sofiafaro-da merged commit 5c12d75 into main Nov 24, 2021

sofiafaro-da deleted the interface-exercise-guard branch November 24, 2021 14:22

sofiafaro-da mentioned this pull request Nov 24, 2021

Error when fetching the wrong template id (via fetch by interface). #11862

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a guard when exercising by interface. #11836

Add a guard when exercising by interface. #11836

sofiafaro-da commented Nov 23, 2021

cocreature left a comment

cocreature Nov 23, 2021

sofiafaro-da Nov 23, 2021

remyhaemmerle-da left a comment

remyhaemmerle-da left a comment

sofiafaro-da commented Nov 24, 2021

Add a guard when exercising by interface. #11836

Add a guard when exercising by interface. #11836

Conversation

sofiafaro-da commented Nov 23, 2021

Pull Request Checklist

cocreature left a comment

Choose a reason for hiding this comment

cocreature Nov 23, 2021

Choose a reason for hiding this comment

sofiafaro-da Nov 23, 2021

Choose a reason for hiding this comment

remyhaemmerle-da left a comment

Choose a reason for hiding this comment

remyhaemmerle-da left a comment

Choose a reason for hiding this comment

sofiafaro-da commented Nov 24, 2021