Change variant JSON encoding, so it is easier to pattern match on it …

…in TypeScript (digital-asset#3882) * Change variant json encoding, adding integration test * Add DamlLfTypeLookup dependencies * Add MetadataReader * Add test WIP * Add serialize test cases * Add serialize test cases, WIP * Test for variant encoding decoding * Solving merge conflicts * Updating roundtrip test * Minor cleanup * Addressing code review comments Add JsonVariant custom matcher * Update specification * Update link * Add test case, WIP * Add proper template key resolution * Got rid of choice record ID resolution, resolving choice type and key type * Fixing logging * Add Contract Key decoding tests * cleanup * cleanup * Update JSON variant encoding tests * Add more contract key JSON decoding tests * Fix variant JSON encoding * Change value predicate to support new variant encoding * Change value predicate to support new variant encoding * Add lookup by contract key test case where contract key contains variant and record Add `requiredResource` to bazel utils CHANGELOG_BEGIN - [JSON API - Experimental] Change variant JSON encoding. The new format is ``{ tag: data-constructor, value: argument }``. For example, if we have: ``data Foo = Bar Int | Baz``, these are all valid JSON encodings for values of type Foo: - ``{"tag": "Bar", "value": 42}`` - ``{"tag": "Baz", "value": {}}`` See digital-asset#3622 - [JSON API - Experimental] Fix ``/contracts/lookup` find by contract key. - [JSON API - Experimental] Fix ``/command/exercise`` to support any LF type as a choice argument. See digital-asset#3390 CHANGELOG_END * minor cleanup * Fix copy/paste * Renaming * Got rid of DAML LF identifier resolution resolving DAML LF Type based on command type * Address code review comments, thanks @S11001001 * Address code review comments, thanks @S11001001 Do not include any error handling here; this partial function should only match the successful case, JsonVariant. * Address code review comments, thanks @S11001001 comment * Address code review comments, thanks @S11001001 using `JsonVariant` for variant encoding/decoding * Address code review comments, thanks @S11001001 replace `find` and `map` chain with collectFirst * Update docs/source/json-api/lf-value-specification.rst Co-Authored-By: Stephen Compall <stephen.compall@daml.com> Co-authored-by: Stephen Compall <scompall@nocandysw.com>
TuranDev · Dec 24, 2019 · f22d52a · f22d52a
1 parent 1794d8a
commit f22d52a
Show file tree

Hide file tree

Showing 32 changed files with 659 additions and 172 deletions.
diff --git a/bazel_tools/runfiles/src/main/scala/com/digitalasset/daml/bazeltools/BazelRunfiles.scala b/bazel_tools/runfiles/src/main/scala/com/digitalasset/daml/bazeltools/BazelRunfiles.scala
@@ -32,6 +32,12 @@ trait BazelRunfiles {
       Paths.get(runfilePath.getOrElse(throw new IllegalArgumentException(path.toString)))
     } else
       path
+
+  def requiredResource(name: String): java.io.File = {
+    val file = new java.io.File(rlocation(name))
+    if (file.exists()) file
+    else throw new IllegalStateException(s"File doest not exist: ${file.getAbsolutePath}")
+  }
 }
 
 object BazelRunfiles extends BazelRunfiles
diff --git a/daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/DarReader.scala b/daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/DarReader.scala
@@ -26,7 +26,7 @@ class DarReader[A](
   import DarReader._
 
   /** Reads an archive from a File. */
-  def readArchiveFromFile(darFile: File) =
+  def readArchiveFromFile(darFile: File): Try[Dar[A]] =
     readArchive(darFile.getName, new ZipInputStream(new FileInputStream(darFile)))
 
   /** Reads an archive from a ZipInputStream. The stream will be closed by this function! */

diff --git a/docs/source/json-api/lf-value-specification.rst b/docs/source/json-api/lf-value-specification.rst
@@ -385,7 +385,7 @@ Variants are expressed as
 
 ::
 
-    { constructor: argument }
+    { tag: constructor, value: argument }
 
 For example, if we have
 
@@ -395,10 +395,10 @@ For example, if we have
 
 These are all valid JSON encodings for values of type Foo::
 
-    {"Bar": 42}
-    {"Baz": {}}
-    {"Quux": null}
-    {"Quux": 42}
+    {"tag": "Bar", "value": 42}
+    {"tag": "Baz", "value": {}}
+    {"tag": "Quux", "value": null}
+    {"tag": "Quux", "value": 42}
 
 Note that DAML data types with named fields are compiled by factoring
 out the record. So for example if we have
@@ -418,13 +418,13 @@ and then, from JSON
 
 ::
 
-    {"Bar": {"f1": 42, "f2": true}}
-    {"Baz": {}}
+    {"tag": "Bar", "value": {"f1": 42, "f2": true}}
+    {"tag": "Baz", "value": {}}
 
 This can be encoded and used in TypeScript, including exhaustiveness
-checking; see `a keyed example`_.
+checking; see `a type refinement example`_.
 
-.. _a keyed example: https://www.typescriptlang.org/play/#src=type%20Foo%20%3D%0D%0A%20%20%20%20%7B%20Bar%3A%20%7B%20f1%3A%20number%2C%20f2%3A%20boolean%20%7D%20%7D%0D%0A%20%20%7C%20%7B%20Baz%3A%20%7B%20f3%3A%20string%20%7D%20%7D%3B%0D%0A%0D%0Afunction%20test(v%3A%20Foo)%20%7B%0D%0A%20%20if%20(%22Bar%22%20in%20v)%20%7B%0D%0A%20%20%20%20console.log(v.Bar.f1%2C%20v.Bar.f2)%3B%0D%0A%20%20%7D%20else%20if%20(%22Baz%22%20in%20v)%20%7B%0D%0A%20%20%20%20console.log(v.Baz.f3)%3B%0D%0A%20%20%7D%20else%20%7B%0D%0A%20%20%20%20const%20_%3A%20never%20%3D%20v%3B%0D%0A%20%20%7D%0D%0A%7D%20%0D%0A
+.. _a type refinement example: https://www.typescriptlang.org/play/#code/C4TwDgpgBAYg9nKBeAsAKCpqBvKwCGA5gFxQBEAQvgE5kA0UAbvgDYCuEpuAZgIykA7NgFsARhGoNuAJlKiELCPgFQAvmvSYAPjjxFSlfAC96TVhy5q1AbnTpubAQGNgASzgrgEAM7AAFIyk8HAAlDiaUN4A7q7ATgAWUAEAdASEYdgRmE743tCGtMRZWE4e3nCKySxwhCnM7BDJfAyMyfUcTdIhthhYmNQQwGzUAj19OXnkVCZFveNlFY3Vta3tEN3F-YPDo8UAJhDc+GwswLN92WXAUAD6ghCMEshMYxpoqkA
 
 Enum
 ****

diff --git a/extractor/src/test/suite/scala/com/digitalasset/extractor/GenMapSpec.scala b/extractor/src/test/suite/scala/com/digitalasset/extractor/GenMapSpec.scala
@@ -50,26 +50,26 @@ class GenMapSpec
       """
         {
           "x" : [
-                  [ { "fst" : 1, "snd" : "1.0" },                            { "Left" : 0 } ]
+                  [ { "fst" : 1, "snd" : "1.0" },                            { "tag" : "Left", "value" : 0 } ]
                 ],
           "party" : "Bob"
         }
       """,
       """
         {
           "x" : [
-                  [ { "fst" : 1, "snd" : "1.0" },                              { "Left" : 0 } ],
-                  [ { "fst" : -2, "snd" : "-2.2222222222" },                   { "Right" : "1.1111111111" } ]
+                  [ { "fst" : 1, "snd" : "1.0" },                              { "tag" : "Left", "value" : 0 } ],
+                  [ { "fst" : -2, "snd" : "-2.2222222222" },                   { "tag" : "Right", "value" : "1.1111111111" } ]
                 ],
           "party" : "Bob"
         }
       """,
       """
         {
           "x" : [
-                  [ { "fst" : 1, "snd" : "1.0" },                               { "Left" : 0 } ],
-                  [ { "fst" : -2, "snd" : "-2.2222222222" },                    { "Right" : "1.1111111111" } ],
-                  [ { "fst" : -3, "snd" : "-3333333333333333333333333333.0" },  { "Right" : "-2.2222222222" } ]
+                  [ { "fst" : 1, "snd" : "1.0" },                               { "tag" : "Left", "value" : 0 } ],
+                  [ { "fst" : -2, "snd" : "-2.2222222222" },                    { "tag" : "Right", "value" : "1.1111111111" } ],
+                  [ { "fst" : -3, "snd" : "-3333333333333333333333333333.0" },  { "tag" : "Right", "value" : "-2.2222222222" } ]
                 ],
           "party" : "Bob"
         }

diff --git a/extractor/src/test/suite/scala/com/digitalasset/extractor/RecordsAndVariantsSpec.scala b/extractor/src/test/suite/scala/com/digitalasset/extractor/RecordsAndVariantsSpec.scala
@@ -45,7 +45,8 @@ class RecordsAndVariantsSpec
           "party" : "Bob",
           "reference" : "All-in-one",
           "deepNested" : {
-            "MaybeRecRecordABRight" : {
+            "tag" : "MaybeRecRecordABRight",
+            "value" : {
               "baz" : {
                 "baz" : false,
                 "foo" : "foo"
@@ -64,7 +65,8 @@ class RecordsAndVariantsSpec
             "foo" : true
           },
           "eitherVariant" : {
-            "RightM" : 7
+            "tag": "RightM",
+            "value" : 7
           },
           "recordTextInt" : {
             "baz" : 6,

diff --git a/language-support/ts/codegen/tests/ts/generated/src/__tests__/test.ts b/language-support/ts/codegen/tests/ts/generated/src/__tests__/test.ts
@@ -106,7 +106,7 @@ test('create + fetch & exercise', async () => {
     polyRecord: {one: '10', two: 'XYZ'},
     imported: {field: {something: 'pqr'}},
     archiveX: {},
-    either: {'Right': 'really?'},
+    either: {tag: 'Right', value: 'really?'},
     tuple: {_1: '12', _2: 'mmm'},
     enum: Main.Color.Red,
     enumList: [Main.Color.Red, Main.Color.Blue, Main.Color.Yellow]

diff --git a/ledger-service/http-json/src/main/scala/com/digitalasset/http/CommandService.scala b/ledger-service/http-json/src/main/scala/com/digitalasset/http/CommandService.scala
@@ -35,7 +35,6 @@ import scala.util.{Failure, Success}
 
 class CommandService(
     resolveTemplateId: PackageService.ResolveTemplateId,
-    resolveChoiceRecordId: PackageService.ResolveChoiceRecordId,
     submitAndWaitForTransaction: LedgerClientJwt.SubmitAndWaitForTransaction,
     submitAndWaitForTransactionTree: LedgerClientJwt.SubmitAndWaitForTransactionTree,
     timeProvider: TimeProvider,
@@ -111,14 +110,11 @@ class CommandService(
     for {
       templateId <- resolveTemplateId(input.templateId)
         .leftMap(e => Error('exerciseCommand, e.shows))
-      choiceRecordId <- resolveChoiceRecordId(templateId, input.choice)
-        .leftMap(e => Error('exerciseCommand, e.shows))
     } yield
       Commands.exercise(
         refApiIdentifier(templateId),
         input.contractId,
         input.choice,
-        choiceRecordId,
         input.argument)
 
   private def submitAndWaitRequest(
@@ -169,12 +165,6 @@ class CommandService(
       .leftMap(e => Error('activeContracts, e.shows))
   }
 
-  private def contracts(response: lav1.command_service.SubmitAndWaitForTransactionResponse)
-    : Error \/ List[Contract[lav1.value.Value]] =
-    response.transaction
-      .toRightDisjunction(Error('contracts, s"Received response without transaction: $response"))
-      .flatMap(Commands.contracts)
-
   private def contracts(response: lav1.command_service.SubmitAndWaitForTransactionTreeResponse)
     : Error \/ List[Contract[lav1.value.Value]] =
     response.transaction

diff --git a/ledger-service/http-json/src/main/scala/com/digitalasset/http/Config.scala b/ledger-service/http-json/src/main/scala/com/digitalasset/http/Config.scala
@@ -7,7 +7,7 @@ import java.io.File
 import java.nio.file.Path
 
 import akka.stream.ThrottleMode
-import com.digitalasset.http.util.ExceptionOps._
+import com.digitalasset.util.ExceptionOps._
 import com.digitalasset.ledger.api.refinements.ApiTypes.ApplicationId
 import scalaz.std.option._
 import scalaz.syntax.traverse._

diff --git a/ledger-service/http-json/src/main/scala/com/digitalasset/http/ContractsFetch.scala b/ledger-service/http-json/src/main/scala/com/digitalasset/http/ContractsFetch.scala
@@ -26,7 +26,7 @@ import com.digitalasset.http.json.JsonProtocol.LfValueDatabaseCodec.{
   apiValueToJsValue => lfValueToDbJsValue
 }
 import com.digitalasset.http.util.IdentifierConverters.apiIdentifier
-import com.digitalasset.http.util.ExceptionOps._
+import com.digitalasset.util.ExceptionOps._
 import com.digitalasset.jwt.domain.Jwt
 import com.digitalasset.ledger.api.v1.transaction.Transaction
 import com.digitalasset.ledger.api.{v1 => lav1}

diff --git a/ledger-service/http-json/src/main/scala/com/digitalasset/http/ContractsService.scala b/ledger-service/http-json/src/main/scala/com/digitalasset/http/ContractsService.scala
@@ -14,7 +14,7 @@ import com.digitalasset.http.json.JsonProtocol.LfValueCodec
 import com.digitalasset.http.query.ValuePredicate
 import com.digitalasset.http.query.ValuePredicate.LfV
 import com.digitalasset.http.util.ApiValueToLfValueConverter
-import com.digitalasset.http.util.ExceptionOps._
+import com.digitalasset.util.ExceptionOps._
 import com.digitalasset.http.util.FutureUtil.toFuture
 import com.digitalasset.http.util.IdentifierConverters.apiIdentifier
 import com.digitalasset.jwt.domain.Jwt
@@ -74,7 +74,7 @@ class ContractsService(
 
       resolvedTemplateId <- toFuture(resolveTemplateId(templateId)): Future[TemplateId.RequiredPkg]
 
-      predicate = isContractKey(keyToTuple(lfKey)) _
+      predicate = isContractKey(lfKey) _
 
       errorOrAc <- searchInMemoryOneTpId(jwt, party, resolvedTemplateId, Map.empty)
         .collect {
@@ -88,14 +88,7 @@ class ContractsService(
     } yield result
 
   private def isContractKey(k: LfValue)(a: domain.ActiveContract[LfValue]): Boolean =
-    a.key.fold(false)(key => keyToTuple(key) == k)
-
-  private def keyToTuple(a: LfValue): LfValue = a match {
-    case lf.value.Value.ValueRecord(_, fields) =>
-      lf.value.Value.ValueRecord(None, fields.map(k => (None, k._2)))
-    case _ =>
-      a
-  }
+    a.key.fold(false)(_ == k)
 
   def findByContractId(
       jwt: Jwt,

diff --git a/ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala b/ledger-service/http-json/src/main/scala/com/digitalasset/http/Endpoints.scala
@@ -11,7 +11,7 @@ import com.digitalasset.daml.lf
 import com.digitalasset.http.Statement.discard
 import com.digitalasset.http.domain.JwtPayload
 import com.digitalasset.http.json.{DomainJsonDecoder, DomainJsonEncoder, ResponseFormats, SprayJson}
-import com.digitalasset.http.util.ExceptionOps._
+import com.digitalasset.util.ExceptionOps._
 import com.digitalasset.http.util.FutureUtil.{either, eitherT}
 import com.digitalasset.http.util.{ApiValueToLfValueConverter, FutureUtil}
 import com.digitalasset.jwt.domain.Jwt
@@ -136,12 +136,17 @@ class Endpoints(
 
         (jwt, jwtPayload, reqBody) = input
 
+        _ = logger.debug(s"/contracts/lookup reqBody: $reqBody")
+
+        // TODO(Leo): decode to domain.ContractLocator[LfValue], findByContractKey converts it to LfValue
         cl <- either(
           decoder
             .decodeContractLocator(reqBody)
             .leftMap(e => InvalidUserInput(e.shows))
         ): ET[domain.ContractLocator[ApiValue]]
 
+        _ = logger.debug(s"/contracts/lookup cl: $cl")
+
         ac <- eitherT(
           handleFutureFailure(contractsService.lookup(jwt, jwtPayload, cl))
         ): ET[Option[domain.ActiveContract[LfValue]]]

diff --git a/ledger-service/http-json/src/main/scala/com/digitalasset/http/HttpService.scala b/ledger-service/http-json/src/main/scala/com/digitalasset/http/HttpService.scala
@@ -21,7 +21,7 @@ import com.digitalasset.http.json.{
   JsValueToApiValueConverter
 }
 import com.digitalasset.http.util.ApiValueToLfValueConverter
-import com.digitalasset.http.util.ExceptionOps._
+import com.digitalasset.util.ExceptionOps._
 import com.digitalasset.http.util.FutureUtil._
 import com.digitalasset.http.util.IdentifierConverters.apiLedgerId
 import com.digitalasset.jwt.JwtDecoder
@@ -104,7 +104,6 @@ object HttpService extends StrictLogging {
 
       commandService = new CommandService(
         packageService.resolveTemplateId,
-        packageService.resolveChoiceRecordId,
         LedgerClientJwt.submitAndWaitForTransaction(client),
         LedgerClientJwt.submitAndWaitForTransactionTree(client),
         TimeProvider.UTC
@@ -214,7 +213,9 @@ object HttpService extends StrictLogging {
 
     val decoder = new DomainJsonDecoder(
       packageService.resolveTemplateId,
-      packageService.resolveChoiceRecordId,
+      packageService.resolveTemplateRecordType,
+      packageService.resolveChoiceRecordType,
+      packageService.resolveKeyType,
       jsValueToApiValueConverter.jsObjectToApiRecord,
       jsValueToApiValueConverter.jsValueToApiValue,
     )