New Ledger API test tool (digital-asset#2282)

* New acceptance test tool prototype * Fix transaction filter helper * Address digital-asset#2282 (comment) * Address digital-asset#2282 (comment) Moreover, creates specific helpers for test DAML code. * Move helpers from session to context Remove a circular dependency between session and context, and limits the usage of implicits. * Run tests with ledgers out of process * Prepare for drop in * Temporary timeout increase * Move bulk of the logic to the test runner * Have a map of tests for CLI integration * Timeout per-test, runner uses context, renaming * Allow usage of time service, integrate semantic tests * Integrate CLI SSL configuration * Integrate failure expectation CLI option * Integrate with failure output verbosity CLI option * Integrate with test listing and picking CLI options * Integrate with timeout scaling CLI option * Integrate with command TTL CLI option * Review CODEOWNERS * Drop previously unimplemented CLI parameters * Expand test todo list to encompass latest additions * Deduplicate some testing infrastructure, swap in new test tool * Remove unused rule * Address digital-asset#2282 (comment) * Address digital-asset#2282 (comment) * Address digital-asset#2282 (comment) * Address digital-asset#2282 (comment) * WIP * Make gRPC thread factory spawn non-deamon threads * Remove debug prints, complete divulgence test (missing assertion messages) * Try to detect assertion failure line number * Add Sandbox binary that starts an ephemeral postgres instance This allows us to use the client_server_test bazel macro to run against the sandbox backed by postgres. Fixes digital-asset#1543 * Add Sandbox binary that starts an ephemeral postgres instance This allows us to use the client_server_test bazel macro to run against the sandbox backed by postgres. Fixes digital-asset#1543 * Address digital-asset#2282 (comment) * Remove TlsConfiguration duplicate, move original to commons * Make assertion more readable Co-Authored-By: Gerolf Seitz <gerolf.seitz@digitalasset.com> * Fix build errors introduced in previous commit - exclude time service tests from reference-v2 - add copyright header to TlsConfiguration * Complete divulgence test and address review comments Restores comments from original divulgence tests and strives to be as close to them as possible. - Address digital-asset#2282 (comment) - Address digital-asset#2282 (comment) - Address digital-asset#2282 (comment) * Delete old divulgence tests * Restore methods to fetch contracts by template identifier * Run conformance tests on in-memory sandbox * Run conformance tests on the persistent sandbox * Remove workflow identifier workaround * New contract keys test (digital-asset#2452) * Set channel type for LedgerSession * Add ContractKeysIT * Run tests against reference-server v2 * Removed the old ContractKeysIT. However, ... ... I cannot remove ContractKeysChecks completely, because we have another test path for DARs created prior to DAML-LF 1.6, which doesn't have the submitter == maintainer restriction for contract keys. * Use codegen instead of custom code * Add missing copyright headers * Drop sandbox conformance test suite (seems to not work on Windows CI) * Make time simpler * Test tool tests to run only semantic tests * Make CI behave more consistently * Add missing files to run tests on Postrgres * Fix flaky divulgence test, introduce readable identifiers * Increase sandbox conformance tests timeout * Address digital-asset#2282 (comment) * Reintroduce test resource extraction * Read semantic tests from the classpath rather than Bazel * Use codegen types for ContractKeys integration test allocateParty returns Primitive.Party and create and exercise also accept Primitive.Party. This way we don't have to keep a String and Primitive.Party representation around. * Bump timeout for sandbox conformance tests * Remove dependency on Bazel runfiles
TuranDev · Aug 9, 2019 · afa4659 · afa4659
1 parent d388c56
commit afa4659
Show file tree

Hide file tree

Showing 83 changed files with 2,058 additions and 1,494 deletions.
diff --git a/CODEOWNERS b/CODEOWNERS
@@ -28,7 +28,7 @@ azure-cron.yml      @garyverhaegen-da
 /daml-lf/validation/    @remyhaemmerle-da
 
 /ledger/      @gerolf-da @rautenrieth-da @stefanobaghino-da
-/ledger-api/  @gerolf-da
+/ledger-api/  @gerolf-da @stefanobaghino-da
 
 # Ecosystems
 /language-support/hs/       @nickchapman-da

diff --git a/daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/UniversalArchiveReader.scala b/daml-lf/archive/src/main/scala/com/digitalasset/daml/lf/archive/UniversalArchiveReader.scala
@@ -88,7 +88,7 @@ object UniversalArchiveReaderWithVersion {
   private def parseDalf(is: InputStream) = Try(Reader.readArchiveAndVersion(is))
 }
 
-private[lf] object SupportedFileType {
+object SupportedFileType {
   def supportedFileType(f: File): Try[SupportedFileType] =
     if (DarFile.matchesFileExtension(f)) Success(DarFile)
     else if (DalfFile.matchesFileExtension(f)) Success(DalfFile)

diff --git a/...testing-tools/src/main/scala/com/digitalasset/daml/lf/engine/testing/SemanticTester.scala b/...testing-tools/src/main/scala/com/digitalasset/daml/lf/engine/testing/SemanticTester.scala
@@ -67,7 +67,9 @@ class SemanticTester(
               body)
             ScenarioRunner(machine, partyNameMangler = partyNameMangler).run() match {
               case Left((err, _ledger @ _)) =>
-                sys.error(s"error running scenario $err in scenario: $qualifiedName")
+                throw new RuntimeException(
+                  s"error running scenario $err in scenario: $qualifiedName",
+                  err)
               case Right((_time @ _, _steps @ _, ledger)) =>
                 qualifiedName -> ledger
             }
@@ -377,7 +379,7 @@ class SemanticTester(
 }
 
 object SemanticTester {
-  case class SemanticTesterError(reference: String, msg: String)
+  final case class SemanticTesterError(reference: String, msg: String)
       extends RuntimeException(s"Error in $reference: $msg", null, true, false)
 
   // Keep in sync with scenario extracting code in SemanticTester class

diff --git a/extractor/BUILD.bazel b/extractor/BUILD.bazel
@@ -22,6 +22,7 @@ compileDependencies = [
     "//ledger-api/rs-grpc-akka",
     "//language-support/scala/bindings",
     "//ledger/ledger-api-client",
+    "//ledger/ledger-api-common",
     "//ledger/ledger-api-domain",
     "//ledger-service/utils",
     "//3rdparty/jvm/io/netty:netty_tcnative_boringssl_static",

diff --git a/ledger-api/testing-utils/BUILD.bazel b/ledger-api/testing-utils/BUILD.bazel
@@ -23,5 +23,6 @@ da_scala_library(
         "//3rdparty/jvm/org/scalatest",
         "//3rdparty/jvm/org/slf4j:slf4j_api",
         "//ledger-api/grpc-definitions:ledger-api-scalapb",
+        "//ledger-api/rs-grpc-akka",
     ],
 )
diff --git a/...sset.platform.testing/LedgerBackend.scala → ...ger/api/testing/utils/LedgerBackend.scala b/...sset.platform.testing/LedgerBackend.scala → ...ger/api/testing/utils/LedgerBackend.scala
@@ -1,7 +1,7 @@
 // Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
-package com.digitalasset.platform.testing
+package com.digitalasset.ledger.api.testing.utils
 
 sealed abstract class LedgerBackend extends Product with Serializable
 

diff --git a/...t.platform.testing/MultiFixtureBase.scala → .../api/testing/utils/MultiFixtureBase.scala b/...t.platform.testing/MultiFixtureBase.scala → .../api/testing/utils/MultiFixtureBase.scala
@@ -1,21 +1,21 @@
 // Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
-package com.digitalasset.platform.testing
+package com.digitalasset.ledger.api.testing.utils
 
 import java.util.concurrent.{Executors, ScheduledExecutorService, TimeUnit}
 
 import com.digitalasset.grpc.adapter.utils.DirectExecutionContext
-import org.scalatest._
 import org.scalatest.concurrent.{AsyncTimeLimitedTests, ScaledTimeSpans}
 import org.scalatest.exceptions.TestCanceledException
 import org.scalatest.time.Span
+import org.scalatest._
 
 import scala.collection.immutable.Iterable
 import scala.concurrent.ExecutionContext.global
 import scala.concurrent.{Future, Promise, TimeoutException}
-import scala.concurrent.duration._
 import scala.util.control.{NoStackTrace, NonFatal}
+import scala.concurrent.duration.DurationInt
 
 trait MultiFixtureBase[FixtureId, TestContext]
     extends Assertions

diff --git a/....platform.testing/MultiResourceBase.scala → ...api/testing/utils/MultiResourceBase.scala b/....platform.testing/MultiResourceBase.scala → ...api/testing/utils/MultiResourceBase.scala
@@ -1,9 +1,8 @@
 // Copyright (c) 2019 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
-package com.digitalasset.platform.testing
+package com.digitalasset.ledger.api.testing.utils
 
-import com.digitalasset.ledger.api.testing.utils.{Resource, SuiteResource}
 import org.scalatest.AsyncTestSuite
 
 import scala.collection.immutable

diff --git a/ledger/api-server-damlonx/reference-v2/BUILD.bazel b/ledger/api-server-damlonx/reference-v2/BUILD.bazel
@@ -85,20 +85,20 @@ client_server_test(
         # to slowly add and fix tests one after the other.
         "--all-tests",
         # "--include {}".format(",".join(conformance_tests_to_run)),
-        "--exclude SemanticTests",
+        "--exclude SemanticTests,DivulgenceIT,TimeIT",
     ],
     data = [
-        "//ledger/ledger-api-integration-tests:SemanticTests.dar",
-        "//ledger/sandbox:Test.dar",
+        "//ledger/test-common:SemanticTests.dar",
+        "//ledger/test-common:Test.dar",
         "@postgresql_dev_env//:all",
         "@postgresql_dev_env//:createdb",
         "@postgresql_dev_env//:initdb",
         "@postgresql_dev_env//:pg_ctl",
     ],
     server = ":ephemeral-postgres-reference-server",
     server_args = [
-        "ledger/ledger-api-integration-tests/SemanticTests.dar",
-        "ledger/sandbox/Test.dar",
+        "ledger/test-common/SemanticTests.dar",
+        "ledger/test-common/Test.dar",
     ],
     tags = [
         "dont-run-on-darwin",

diff --git a/ledger/api-server-damlonx/reference/BUILD.bazel b/ledger/api-server-damlonx/reference/BUILD.bazel
@@ -52,13 +52,13 @@ client_server_test(
         "--include=SemanticTests,ContractKeysIT",
     ],
     data = [
-        "//ledger/ledger-api-integration-tests:SemanticTests.dar",
-        "//ledger/sandbox:Test.dar",
+        "//ledger/test-common:SemanticTests.dar",
+        "//ledger/test-common:Test.dar",
     ],
     server = ":reference",
     server_args = [
-        "$(rlocation $TEST_WORKSPACE/$(rootpath //ledger/ledger-api-integration-tests:SemanticTests.dar))",
-        "$(rlocation $TEST_WORKSPACE/$(rootpath //ledger/sandbox:Test.dar))",
+        "$(rlocation $TEST_WORKSPACE/$(rootpath //ledger/test-common:SemanticTests.dar))",
+        "$(rlocation $TEST_WORKSPACE/$(rootpath //ledger/test-common:Test.dar))",
     ],
     tags = [
         # NOTE(JM): As this test is somewhat heavy and has timeouts, run it without competition to avoid flakyness.

diff --git a/ledger/ledger-api-common/BUILD.bazel b/ledger/ledger-api-common/BUILD.bazel
@@ -24,6 +24,7 @@ da_scala_library(
         "//3rdparty/jvm/com/google/guava",
         "//3rdparty/jvm/com/typesafe/akka:akka_stream",
         "//3rdparty/jvm/com/typesafe/scala_logging",
+        "//3rdparty/jvm/io/grpc:grpc_netty",
         "//3rdparty/jvm/io/grpc:grpc_services",
         "//3rdparty/jvm/io/zipkin/brave",
         "//3rdparty/jvm/org/scalaz:scalaz_core",

diff --git a/...set/ledger/api/tls/TlsConfiguration.scala → ...set/ledger/api/tls/TlsConfiguration.scala b/...set/ledger/api/tls/TlsConfiguration.scala → ...set/ledger/api/tls/TlsConfiguration.scala
diff --git a/ledger/ledger-api-integration-tests/BUILD.bazel b/ledger/ledger-api-integration-tests/BUILD.bazel
@@ -69,21 +69,6 @@ java_library(
     ],
 )
 
-da_scala_library(
-    name = "ledger-api-integration-tests-as-library",
-    srcs = glob([
-        "src/test/itsuite/**/*.scala",
-        "src/test/semanticsuite/**/*.scala",
-    ]),
-    visibility = [
-        "//ledger/ledger-api-test-tool:__subpackages__",
-    ],
-    deps = [
-        ":ledger-api-integration-tests-lib",
-        "//daml-lf/testing-tools",
-    ] + dependencies,
-)
-
 da_scala_test_suite(
     name = "ledger-api-integration-tests",
     size = "large",
@@ -98,8 +83,8 @@ da_scala_test_suite(
         # exclude = ["src/test/itsuite/**/TransactionServiceIT.scala"],
     ),
     data = [
-        "//ledger/sandbox:Test.dar",
         "//ledger/sandbox:Test-1.5.dar",
+        "//ledger/test-common:Test.dar",
         "@postgresql_dev_env//:all",
         "@postgresql_dev_env//:createdb",
         "@postgresql_dev_env//:initdb",
@@ -118,16 +103,3 @@ da_scala_test_suite(
         ":ledger-api-integration-tests-lib",
     ] + dependencies,
 )
-
-TEST_FILES = [
-    "SemanticTests",
-]
-
-[
-    daml_compile(
-        name = name,
-        main_src = "src/test/semanticsuite/daml/%s.daml" % name,
-        visibility = ["//visibility:public"],
-    )
-    for name in TEST_FILES
-]