/benchmark plutus-benchmark:validation

^ just out of curiosity

Comparing benchmark results of 'plutus-benchmark:validation' on '61a9a0fbe' (base) and 'd3b87a894' (PR)

Yup, makes perfect sense. It might be possible to tame some of that, though.

/benchmark plutus-benchmark:validation

Comparing benchmark results of 'plutus-benchmark:validation' on '61a9a0fbe' (base) and '7d03746ff' (PR)

Oh.

A pleasingly small change, shame about the appalling performance 😅

We also do need a way to be able to pick this behaviour at runtime (for now)...

We also do need a way to be able to pick this behaviour at runtime (for now)...

Implemented a proof of concept in this commit. Proof that it works.

Neat. I had to stare at it for quite a while, but seems nice. Any idea why the performance is so garbage?

/benchmark plutus-benchmark:validation

I had to stare at it for quite a while, but seems nice.

I posted a mini version of this problem as a challenge here (more than a year ago).

Any idea why the performance is so garbage?

No point to debug it before the monomorphic makeKnown PR is merged (and some form of inlining of toBuiltinsRuntime for that matter).

Comparing benchmark results of 'plutus-benchmark:validation' on '61a9a0fbe' (base) and 'f7ed238a8' (PR)

Hmm, I'm keen to get this moving, though. It seems like the effect is big enough that surely it should be observable what's up even without the other things being done yet?

Hmm, I'm keen to get this moving, though.

Realistically, it's unlikely to happen within the next several weeks. I'll update the makeKnown monomorphization PR, which will probably change the numbers (simply because they got so much worse when I added support for emitting), but all this deferring is not free and it happens at runtime, so we gonna get a slowdown. Theoretically (theoretically!) deferring can be made even faster than the baseline (probably negligibly so, though), but for that we need to convert TypeScheme to a RuntimeScheme statically, which is only possible if TypeScheme is defined differently. I.e. there's a lot to do for this PR to become fast.

What would be considered an acceptable slowdown? Up to 15? 10? 5?

It seems like the effect is big enough that surely it should be observable what's up even without the other things being done yet?

Let's talk about that once the makeKnown monomorphization PR is in.

/benchmark plutus-benchmark:validation

Comparing benchmark results of 'plutus-benchmark:validation' on 'edcf0e886' (base) and '2bf38b98b' (PR)

/benchmark plutus-benchmark:validation

Comparing benchmark results of 'plutus-benchmark:validation' on 'edcf0e886' (base) and '42357c11c' (PR)

^ @michaelpj told ya. The average is +10.8%, which is not that bad given that this is truly a less efficient way of doing builtin application and we've got similar slowdowns in the past by tweaking unlifting just a little bit.

Still, I believe we can optimize what's in here, but not without substantial effort (in particular changing TypeScheme and how RuntimeScheme is computed from it and getting the inlining of toBuiltinsRuntime PR merged).

Comparing benchmark results of 'plutus-benchmark:validation' on 'edcf0e886' (base) and 'f841f1b1a' (PR)

/benchmark plutus-benchmark:validation

Comparing benchmark results of 'plutus-benchmark:validation' on 'edcf0e886' (base) and 'bbb1ac993' (PR)

The average is +10.8%, which is not that bad given that this is truly a less efficient way of doing builtin application and we've got similar slowdowns in the past by tweaking unlifting just a little bit.

Okay, I'm a little confused, lots of benchmarks. If I understand correctly, the slowdown applies to both ways of doing unlifting, and is a consequence of the machinery, rather than one way being especially slower or faster?

A relevant question might be: if we dropped the need to have this be configurable, i.e. we just switched to the new way, would that make things faster (we might have this option later)?

Okay, I'm a little confused, lots of benchmarks. I

Yeah, sorry, I should've been clearer.

If I understand correctly, the slowdown applies to both ways of doing unlifting, and is a consequence of the machinery, rather than one way being especially slower or faster?

Yes, the difference is like 1%, i.e. probably noise.

A relevant question might be: if we dropped the need to have this be configurable, i.e. we just switched to the new way, would that make things faster (we might have this option later)?

I don't think so. It's not really configurable right now, it's a proof-of-concept. That matching on UnliftMode is really supposed to be optimized away by the case-of-known-constructor optimization pass. Once we thread that UnliftMode parameter through the whole evaluation pipeline, the builtlin application is going to be come a little slower that what it's now in this PR.

Once we thread that UnliftMode parameter through the whole evaluation pipeline, the builtlin application is going to be come a little slower that what it's now in this PR.

Yep I figured.

/benchmark plutus-benchmark:validation

Comparing benchmark results of 'plutus-benchmark:validation' on 'dc9275462' (base) and 'e3073ec34' (PR)

Seems like rebasing on master made things a tiny bit worse, by about 1% or so. Maybe just noise.

Closing in favor of #4516.