Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use libunwind in jemalloc heap profiling #19156

Merged
merged 11 commits into from
Jun 9, 2023

Conversation

neunhoef
Copy link
Member

@neunhoef neunhoef commented Jun 2, 2023

Use libunwind in jemalloc heap profiling if available.

This is necessary to get jemalloc heap profiling to work with libmusl
builds and static binaries.

This implements the following:

If USE_LIBUNWIND is on, then we add a dependency in the build process
such that jemalloc_build can rely on the fact that libunwind is already
built. Furthermore, the configure script of jemalloc gets the newly
built libunwind.a and the included headers and can then be switched to
libunwind profiling mode.

Scope & Purpose

  • [*] 💩 Bugfix

Checklist

There is no enterprise part.

@neunhoef neunhoef added this to the devel milestone Jun 2, 2023
@neunhoef neunhoef self-assigned this Jun 2, 2023
@cla-bot cla-bot bot added the cla-signed label Jun 2, 2023
@neunhoef neunhoef requested review from markuspf and jsteemann June 2, 2023 10:57
@@ -757,7 +757,7 @@ endfunction()
# JEMALLOC
# ------------------------------------------------------------------------------

option(USE_JEMALLOC_PROF "use jemalloc profiler" OFF)
option(USE_JEMALLOC_PROF "use jemalloc profiler" ON)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to turn on profiling by default?

Copy link
Contributor

@jsteemann jsteemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if this PR does not enable the profiling code by default.
I can't estimate the effects of including the profiling code on runtime performance.
If we want to turn on the profiling code by default, we should first run a side-by-side performance comparison of standard devel with a branch that turns the profiling code on, and then see if the performance results are "ok enough" to use that.

Comment on lines 35 to 42
if (USE_JEMALLOC_PROF)
SET(JEMALLOC_PROF "--enable-prof")
if (USE_LIBUNWIND)
SET(JEMALLOC_PROF "--enable-prof" "--enable-prof-libunwind" "--with-static-libunwind=${LIBUNWIND_LIB}")
SET(JEMALLOC_C_FLAGS "${CMAKE_C_FLAGS} -I${LIBUNWIND_HOME}/include")
else ()
SET(JEMALLOC_PROF "--enable-prof")
endif()
endif ()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think set to the CMAKE_C_FLAGS will be ok, because we're in jemalloc cmakelists.txt.
But it doesn't really matter

@neunhoef
Copy link
Member Author

neunhoef commented Jun 7, 2023

Here is some evidence that compares a build without profiling with one with profiling (inactive) and with profiling (active) from our simple performance tests:
comparison.txt

And here is some data from a program which does only allocations and deallocations in many threads:

jemalloc, no prof, libmusl, static:
  4.27  4.29  4.30  4.29  4.28  4.29  4.32  4.27  4.31  4.32
Median 4.29

jemalloc, prof compiled in, switched off, libmusl, static:
  4.29  4.26  4.32  4.31  4.31  4.32  4.28  4.30  4.29  4.36
Median 4.30

jemalloc, prof compiled in, switched on, libmusl, static:
  4.93  4.96  4.98  4.94  4.92  4.98  4.98  5.06  4.92  4.99
Median 4.96

@neunhoef
Copy link
Member Author

neunhoef commented Jun 7, 2023

Conclusion: Compiling in profiling support does not cost any measurable performance, not even in a program which does lots of allocations in many threads. Activating it can cost some performance, but even in a program which only allocates and frees the overhead is some 15%.
Therefore this PR switches USE_JEMALLOC_PROF by default to 1.

Copy link
Contributor

@markuspf markuspf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and I agree that there is no discernible performance penalty when compiling profiling in as a default.

There are big upsides in being able to profile without having to recompile!

@neunhoef neunhoef merged commit 16c22d2 into devel Jun 9, 2023
@neunhoef neunhoef deleted the bug-fix/use-libunwind-in-jemalloc-prof branch June 9, 2023 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants