Skip to content

Commit

Permalink
Implement a simple profiler for DAML scenarios (digital-asset#5957)
Browse files Browse the repository at this point in the history
* Implement a simple profiler for DAML scenarios

The profiler runs a single scenario and records timing information when
each function (and some other closures) are entered and left. The
resulting information can be visualized as a flamegraph using
[speedscope](https://www.speedscope.app/).

The profiler works by instrumenting the CEK machine at the heart of
DAML Engine. Unfortunetaly, this causes a very small overhead on
non-profiling runs too. However, in my benchmarks I could not measure
any significant impact on the overall runtime at all. More precisely,
the overhead is as follows:

Every closure now has an additional field called `label`. In
non-profiling runs this field is always set to `null`. This field needs
to be allocated, copied whenever we copy a closure and scanned during
garbage collection. Additionally, whenever we enter a closure, we check
this field and whenever it is _not_ `null`, i.e. never during
non-profiling runs, we record an "open event" and set up a hook for the
corresponding "close event". Thus, the additional cost during
non-profiling runs are a single pointer comparison and a jump beyond
the "then branch".

Since this is still very much in active development, there are no
documentation, other than an entry in a README, and no tests yet. They
will come before we promote this. However, the UX will look very
different then since we already have plans to significantly change it.

CHANGELOG_BEGIN
CHANGELOG_END

* Run scalafmt

* Make profiling argument to PureCompiledPackges optional

* Fix a bunch of tests

CHANGELOG_BEGIN
CHANGELOG_END

* scalafmt is so annoying

CHANGELOG_BEGIN
CHANGELOG_END

* Apply simple suggestions

CHANGELOG_BEGIN
CHANGELOG_END
  • Loading branch information
hurryabit authored May 18, 2020
1 parent c94f7e1 commit caedc72
Show file tree
Hide file tree
Showing 20 changed files with 582 additions and 198 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,8 @@ class ReplService(
respObs: StreamObserver[LoadPackageResponse]): Unit = {
val (pkgId, pkg) = Decode.decodeArchiveFromInputStream(req.getPackage.newInput)
packages = packages + (pkgId -> pkg)
compiledDefinitions = compiledDefinitions ++ Compiler(packages).unsafeCompilePackage(pkgId)
compiledDefinitions = compiledDefinitions ++ Compiler(packages, Compiler.NoProfile)
.unsafeCompilePackage(pkgId)
respObs.onNext(LoadPackageResponse.newBuilder.build)
respObs.onCompleted()
}
Expand Down Expand Up @@ -218,8 +219,9 @@ class ReplService(
}

val allPkgs = packages + (homePackageId -> pkg)
val defs = Compiler(allPkgs).unsafeCompilePackage(homePackageId)
val compiledPackages = PureCompiledPackages(allPkgs, compiledDefinitions ++ defs)
val defs = Compiler(allPkgs, Compiler.NoProfile).unsafeCompilePackage(homePackageId)
val compiledPackages =
PureCompiledPackages(allPkgs, compiledDefinitions ++ defs, Compiler.NoProfile)
val runner = new Runner(
compiledPackages,
Script.Action(scriptExpr, ScriptIds(scriptPackageId)),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ class Context(val contextId: Context.ContextId) {
// if any change we recompile everything
extPackages --= unloadPackages
extPackages ++= newPackages
extDefns = assert(Compiler.compilePackages(extPackages))
extDefns = assert(Compiler.compilePackages(extPackages, Compiler.NoProfile))
modDefns = HashMap.empty
modules.values
} else {
Expand All @@ -118,7 +118,7 @@ class Context(val contextId: Context.ContextId) {
}

val pkgs = allPackages
val compiler = Compiler(pkgs)
val compiler = Compiler(pkgs, Compiler.NoProfile)

modulesToCompile.foreach { mod =>
if (!omitValidation)
Expand Down Expand Up @@ -152,7 +152,7 @@ class Context(val contextId: Context.ContextId) {
Speedy.Machine
.build(
sexpr = defn,
compiledPackages = PureCompiledPackages(allPackages, defns),
compiledPackages = PureCompiledPackages(allPackages, defns, Compiler.NoProfile),
submissionTime,
initialSeeding,
Set.empty,
Expand Down
23 changes: 23 additions & 0 deletions daml-lf/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,29 @@ $ bazel run //:daml-lf-repl -- test Project.tests $PWD/project.dalf

NOTE: When running via `bazel run` one needs to specify full path (or relative path from repo root), since Bazel runs all commands from repository root.

Profiling scenarios
-------------------

DAML-LF-REPL provides a command to run a scenario and collect profiling
information while running it. This information is then written into a file that
can be viewed using the [speedscope](https://www.speedscope.app/) flamegraph
visualizer. The easiest way to install speedscope is to run
```shell
$ npm install -g speedscope
```
See the [Offline usage](https://github.com/jlfwong/speedscope#usage) section of
its documentation for alternatives.

Once speedscope is installed, the profiler can be invoked via
```shell
$ bazel run //:daml-lf-repl -- profile Module.Name:scenarioName /path/to.dar /path/to/output.json
```
and the profile viewed via
```shell
$ speedscope /path/to/output.json
```


Scala house rules
-----------------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ final class ConcurrentCompiledPackages extends MutableCompiledPackages {
def getDefinition(dref: speedy.SExpr.SDefinitionRef): Option[speedy.SExpr] =
Option(_defns.get(dref))

def profilingMode = speedy.Compiler.NoProfile

/** Might ask for a package if the package you're trying to add references it.
*
* Note that when resuming from a [[Result]] the continuation will modify the
Expand Down Expand Up @@ -80,7 +82,9 @@ final class ConcurrentCompiledPackages extends MutableCompiledPackages {
pkgId, { _ =>
// Compile the speedy definitions for this package.
val defns =
speedy.Compiler(packages orElse state.packages).unsafeCompilePackage(pkgId)
speedy
.Compiler(packages orElse state.packages, profilingMode)
.unsafeCompilePackage(pkgId)
defns.foreach {
case (defnId, defn) => _defns.put(defnId, defn)
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ final class Engine {
) {
val machine = Machine
.build(
sexpr = speedy.Compiler(compiledPackages.packages).unsafeCompile(commands),
sexpr = compiledPackages.compiler.unsafeCompile(commands),
compiledPackages = compiledPackages,
submissionTime = submissionTime,
seeds = seeding,
Expand Down
1 change: 1 addition & 0 deletions daml-lf/interpreter/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ da_scala_library(
"//daml-lf/transaction",
"//daml-lf/validation",
"@maven//:com_google_protobuf_protobuf_java",
"@maven//:io_spray_spray_json_2_12",
"@maven//:org_scalaz_scalaz_core_2_12",
"@maven//:org_slf4j_slf4j_api",
"@maven//:org_typelevel_paiges_core_2_12",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ object PlaySpeedy {

def main(args0: List[String]) = {
val config: Config = parseArgs(args0)
val compiler: Compiler = Compiler(Map.empty)
val compiler: Compiler = Compiler(Map.empty, Compiler.NoProfile)

val e: SExpr = compiler.unsafeClosureConvert(examples(config.exampleName))
val m: Machine = makeMachine(e)
Expand Down Expand Up @@ -58,8 +58,8 @@ object PlaySpeedy {
private val txSeed = crypto.Hash.hashPrivateKey("SpeedyExplore")

def makeMachine(sexpr: SExpr): Machine = {
val compiledPackages: CompiledPackages = PureCompiledPackages(Map.empty).right.get
val compiler = Compiler(compiledPackages.packages)
val compiledPackages: CompiledPackages =
PureCompiledPackages(Map.empty, Compiler.NoProfile).right.get
Machine.fromSExpr(
sexpr,
compiledPackages,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,21 @@ trait CompiledPackages {
def packageIds: Set[PackageId]
def definitions: PartialFunction[SDefinitionRef, SExpr] =
Function.unlift(this.getDefinition)

def profilingMode: Compiler.ProfilingMode

def compiler: Compiler = Compiler(packages, profilingMode)
}

final class PureCompiledPackages private (
packages: Map[PackageId, Package],
defns: Map[SDefinitionRef, SExpr],
profiling: Compiler.ProfilingMode,
) extends CompiledPackages {
override def packageIds: Set[PackageId] = packages.keySet
override def getPackage(pkgId: PackageId): Option[Package] = packages.get(pkgId)
override def getDefinition(dref: SDefinitionRef): Option[SExpr] = defns.get(dref)
override def profilingMode = profiling
}

object PureCompiledPackages {
Expand All @@ -38,12 +44,16 @@ object PureCompiledPackages {
private[lf] def apply(
packages: Map[PackageId, Package],
defns: Map[SDefinitionRef, SExpr],
profiling: Compiler.ProfilingMode
): PureCompiledPackages =
new PureCompiledPackages(packages, defns)
new PureCompiledPackages(packages, defns, profiling)

def apply(packages: Map[PackageId, Package]): Either[String, PureCompiledPackages] =
def apply(
packages: Map[PackageId, Package],
profiling: Compiler.ProfilingMode = Compiler.NoProfile,
): Either[String, PureCompiledPackages] =
Compiler
.compilePackages(packages)
.map(new PureCompiledPackages(packages, _))
.compilePackages(packages, profiling)
.map(new PureCompiledPackages(packages, _, profiling))

}
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ object Classify { // classify the machine state w.r.t what step occurs next
case SEBuiltinRecursiveDefinition(_) => counts.ebuiltinrecursivedefinition += 1
case SECatch(_, _, _) => counts.ecatch += 1
case SEAbs(_, _) => //never expect these!
case SELabelClosure(_, _) => ()
case SEImportValue(_) => counts.eimportvalue += 1
case SEWronglyTypeContractId(_, _, _) => counts.ewronglytypedcontractid += 1
}
Expand All @@ -113,6 +114,7 @@ object Classify { // classify the machine state w.r.t what step occurs next
case KMatch(_) => counts.kmatch += 1
case KCatch(_, _, _) => counts.kcatch += 1
case KFinished => counts.kfinished += 1
case KLabelClosure(_) | KLeaveClosure(_) => ()
}
}

Expand Down
Loading

0 comments on commit caedc72

Please sign in to comment.