Run Transaction Service IT as part of Ledger API Test Tool (digital-a…

…sset#1434) Add tTansactionServiceIT into Ledger API Test Tool as an optional test. Available tests can be listed with the --list option. Tests can be included with --include, and excluded with --exclude. Fixes digital-asset#1372, digital-asset#1472
kgeonw · Jun 6, 2019 · b61203d · b61203d
1 parent 217c56d
commit b61203d
Show file tree

Hide file tree

Showing 25 changed files with 433 additions and 227 deletions.
diff --git a/docs/source/daml-integration-kit/index.rst b/docs/source/daml-integration-kit/index.rst
@@ -397,17 +397,17 @@ implementation of the :doc:`Ledger API
 </app-dev/index>`. For example, it will show you if
 there are consistency or conformance problem with your implementation.
 
-Assuming that your Ledger API endpoint is accessible at ``localhost:6864``, you can use the tool in the following manner:
+Assuming that your Ledger API endpoint is accessible at ``localhost:6865``, you can use the tool in the following manner:
 
 #. Obtain the tool:
 
    ``curl -L 'https://bintray.com/api/v1/content/digitalassetsdk/DigitalAssetSDK/com/daml/ledger/testtool/ledger-api-test-tool_2.12/$latest/ledger-api-test-tool_2.12-$latest.jar?bt_package=sdk-components' -o ledger-api-test-tool.jar``
 
-#. Obtain the DAML archive required to run the tests:
+#. Obtain the DAML archives required to run the tests:
 
    ``java -jar ledger-api-test-tool.jar --extract``
 
-#. Load ``SemanticTests.dar`` which was created in the current directory into your Ledger.
+#. Load all ``.dar`` files extracted in the current directory into your Ledger.
 
 #. Run the tool against your ledger:
 

diff --git a/docs/source/support/release-notes.rst b/docs/source/support/release-notes.rst
@@ -58,6 +58,17 @@ Sandbox
 - Added recovery around failing ledger entry persistence queries using Postgres. See `#1505 <https://github.com/digital-asset/daml/pull/1505>`__.
 
 
+DAML Integration Kit
+~~~~~~~~~~~~~~~~~~~~
+
+- The :doc:`Ledger API Test Tool </tools/ledger-api-test-tool/index>` can now optionally run ``TransactionServiceIT`` as part of the conformance tests.
+  This means you need to load additional ``.dar`` files into the ledger under test. Please refer to the updated instructions in the :doc:`documentation </tools/ledger-api-test-tool/index>`.
+- Added new CLI options to the :doc:`Ledger API Test Tool </tools/ledger-api-test-tool/index>`:
+
+  - ``--list`` prints all available tests to the console
+  - ``--include`` takes a comma-separated list of test names that should be run
+  - ``--exclude`` takes a comma-separated list of test names that should not be run
+
 0.12.22 - 2019-05-29
 --------------------
 

diff --git a/docs/source/tools/ledger-api-test-tool/index.rst b/docs/source/tools/ledger-api-test-tool/index.rst
@@ -35,21 +35,21 @@ Run the following command to fetch the tool:
 
 This will create a file ``ledger-api-test-tool.jar`` in your current directory.
 
-Extracting ``.dar`` file required to run the tests
+Extracting ``.dar`` files required to run the tests
 ======================================================
 
 Before you can run the Ledger API test tool on your ledger, you need to load a
 specific set of DAML templates onto your ledger.
 
-#. To obtain the corresponding ``.dar`` file, run:
+#. To obtain the corresponding ``.dar`` files, run:
 
    .. code-block:: console
 
      $ java -jar ledger-api-test-tool.jar --extract
 
-   This creates a file ``SemanticTests.dar`` in the current directory.
+   This writes all ``.dar`` files required for the tests into the current directory.
 
-#. Load ``SemanticTests.dar`` into your Ledger.
+#. Load all ``.dar`` files into your Ledger.
 
 Running the tool against a custom Ledger API endpoint
 =====================================================
@@ -79,7 +79,22 @@ Run the tool with ``--help`` flag to obtain the list of options the tool provide
 
    $ java -jar ledger-api-test-tool.jar  --help
 
-|
+Filtering tests
+~~~~~~~~~~~~~~~
+
+You can list the available tests with the ``--list`` flag. Some tests are not run by default. You can run them with the ``--include`` flag. To exclude tests, use the ``--exclude`` flag.
+
+This command only runs the test ``TestA``:
+
+.. code-block:: console
+
+   $ java -jar ledger-api-test-tool.jar --include TestA
+
+This command runs all tests except the test ``TestB``:
+
+.. code-block:: console
+
+   $ java -jar ledger-api-test-tool.jar --exclude TestB
 
 Try out the Ledger API Test Tool against DAML Sandbox
 =====================================================
@@ -90,7 +105,7 @@ If you wanted to test out the tool, you can run it against :doc:`DAML Sandbox
    .. code-block:: console
 
      $ java -jar ledger-api-test-tool.jar --extract
-     $ da sandbox -- SemanticTests.dar
+     $ da sandbox -- *.dar
      $ java -jar ledger-api-test-tool.jar
 
 This should always succeed, as the Sandbox is tested to correctly implement the

diff --git a/...tils/src/main/scala/com/digitalasset/ledger/api/testing/utils/AkkaBeforeAndAfterAll.scala b/...tils/src/main/scala/com/digitalasset/ledger/api/testing/utils/AkkaBeforeAndAfterAll.scala
@@ -19,7 +19,7 @@ trait AkkaBeforeAndAfterAll extends BeforeAndAfterAll {
   protected def actorSystemName = this.getClass.getSimpleName
   private val logger = LoggerFactory.getLogger(getClass)
 
-  private val executorContext = ExecutionContext.fromExecutorService(
+  private lazy val executorContext = ExecutionContext.fromExecutorService(
     Executors.newSingleThreadExecutor(
       new ThreadFactoryBuilder()
         .setDaemon(true)
@@ -28,10 +28,10 @@ trait AkkaBeforeAndAfterAll extends BeforeAndAfterAll {
           logger.error(s"got an uncaught exception on thread: ${thread.getName}"))
         .build()))
 
-  protected implicit val system: ActorSystem =
+  protected implicit lazy val system: ActorSystem =
     ActorSystem(actorSystemName, defaultExecutionContext = Some(executorContext))
 
-  protected implicit val materializer: ActorMaterializer = ActorMaterializer()
+  protected implicit lazy val materializer: ActorMaterializer = ActorMaterializer()
 
   override protected def afterAll(): Unit = {
     materializer.shutdown()

diff --git a/...ala/com/digitalasset/platform/tests/integration/ledger/api/ActiveContractsServiceIT.scala b/...ala/com/digitalasset/platform/tests/integration/ledger/api/ActiveContractsServiceIT.scala
@@ -27,7 +27,6 @@ import org.scalatest.concurrent.{AsyncTimeLimitedTests, ScalaFutures}
 import org.scalatest.time.SpanSugar._
 import org.scalatest.time.{Millis, Span}
 import org.scalatest.{Assertion, AsyncWordSpec, Matchers, OptionValues}
-
 import scalaz.syntax.tag._
 
 /**
@@ -46,11 +45,13 @@ class ActiveContractsServiceIT
     with ScalaFutures
     with AsyncTimeLimitedTests
     with Matchers
-    with OptionValues
-    with TestCommands {
+    with OptionValues {
 
   override def timeLimit: Span = 60.seconds
 
+  protected val testCommands = new TestCommands(config)
+  protected val templateIds = testCommands.templateIds
+
   override implicit def patienceConfig: PatienceConfig =
     PatienceConfig(scaled(Span(30000, Millis)), scaled(Span(500, Millis)))
 
@@ -97,7 +98,7 @@ class ActiveContractsServiceIT
     }.size should equal(occurrence)
 
   def threeCommands(ledgerId: domain.LedgerId, commandId: String): SubmitAndWaitRequest =
-    super.dummyCommands(ledgerId, commandId, "Alice").toWait
+    testCommands.toWait(testCommands.dummyCommands(ledgerId, commandId, "Alice"))
 
   private def filter = TransactionFilter(Map(config.parties.head -> Filters()))
 
@@ -190,11 +191,13 @@ class ActiveContractsServiceIT
           contractId = extractContractId(responses1)
           _ <- submitRequest(
             ctx,
-            buildRequest(
-              ctx.ledgerId,
-              "exercise-test-exercised",
-              Seq(exerciseWithUnit(templateIds.dummy, contractId, "DummyChoice1")),
-              "Alice").toWait)
+            testCommands.toWait(
+              testCommands.buildRequest(
+                ctx.ledgerId,
+                "exercise-test-exercised",
+                Seq(testCommands.exerciseWithUnit(templateIds.dummy, contractId, "DummyChoice1")),
+                "Alice"))
+          )
           responses2 <- waitForActiveContracts(
             ctx.acsService,
             ctx.ledgerId,
@@ -223,11 +226,12 @@ class ActiveContractsServiceIT
         val resultsF = for {
           _ <- submitRequest(
             ctx,
-            buildRequest(
-              ctx.ledgerId,
-              "commandId1",
-              Seq(createWithOperator(templateIds.dummy, "Alice")),
-              "Alice").toWait)
+            testCommands.toWait(
+              testCommands.buildRequest(
+                ctx.ledgerId,
+                "commandId1",
+                Seq(testCommands.createWithOperator(templateIds.dummy, "Alice")),
+                "Alice")))
           responses1 <- waitForActiveContracts(
             ctx.acsService,
             ctx.ledgerId,
@@ -236,11 +240,12 @@ class ActiveContractsServiceIT
           offset = extractOffset(responses1)
           _ <- submitRequest(
             ctx,
-            buildRequest(
-              ctx.ledgerId,
-              "commandId2",
-              Seq(createWithOperator(templateIds.dummyWithParam, "Alice")),
-              "Alice").toWait
+            testCommands.toWait(
+              testCommands.buildRequest(
+                ctx.ledgerId,
+                "commandId2",
+                Seq(testCommands.createWithOperator(templateIds.dummyWithParam, "Alice")),
+                "Alice"))
           )
           responses2 <- transactionClient(ctx)
             .getTransactions(
@@ -283,8 +288,12 @@ class ActiveContractsServiceIT
     "multi-party request comes" should {
       "return the correct set of related contracts" in allFixtures { ctx =>
         val resultsF = for {
-          _ <- submitRequest(ctx, dummyCommands(ctx.ledgerId, "acsCommand-1", "Alice").toWait)
-          _ <- submitRequest(ctx, dummyCommands(ctx.ledgerId, "acsCommand-2", "Bob").toWait)
+          _ <- submitRequest(
+            ctx,
+            testCommands.toWait(testCommands.dummyCommands(ctx.ledgerId, "acsCommand-1", "Alice")))
+          _ <- submitRequest(
+            ctx,
+            testCommands.toWait(testCommands.dummyCommands(ctx.ledgerId, "acsCommand-2", "Bob")))
           allContractsForAlice <- waitForActiveContracts(
             ctx.acsService,
             ctx.ledgerId,

diff --git a/...t/itsuite/scala/com/digitalasset/platform/tests/integration/ledger/api/DivulgenceIT.scala b/...t/itsuite/scala/com/digitalasset/platform/tests/integration/ledger/api/DivulgenceIT.scala
@@ -45,10 +45,12 @@ class DivulgenceIT
     with ScalaFutures
     with AsyncTimeLimitedTests
     with Matchers
-    with OptionValues
-    with TestTemplateIds {
+    with OptionValues {
   override protected def config: Config = Config.default
 
+  protected val testTemplateIds = new TestTemplateIds(config)
+  protected val templateIds = testTemplateIds.templateIds
+
   private implicit def party(s: String): Ref.Party = Ref.Party.assertFromString(s)
   private implicit def pkgId(s: String): Ref.PackageId = Ref.PackageId.assertFromString(s)
   private implicit def id(s: String): Ref.Name = Ref.Name.assertFromString(s)

diff --git a/...tsuite/scala/com/digitalasset/platform/tests/integration/ledger/api/LotsOfPartiesIT.scala b/...tsuite/scala/com/digitalasset/platform/tests/integration/ledger/api/LotsOfPartiesIT.scala
@@ -27,8 +27,10 @@ class LotsOfPartiesIT
     with MultiLedgerFixture
     with SuiteResourceManagementAroundAll
     with AsyncTimeLimitedTests
-    with Matchers
-    with TestTemplateIds {
+    with Matchers {
+
+  protected lazy val testTemplateIds = new TestTemplateIds(config)
+  protected lazy val templateIds = testTemplateIds.templateIds
 
   private val numParties = 1024
   private val allParties: List[String] =