Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an experimental profiling API #1657

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from
Draft

Conversation

fsimonis
Copy link
Member

@fsimonis fsimonis commented May 12, 2023

Main changes of this PR

This PR adds an experimental profiling API to preCICE.

In user/adapter-code, one can call pushProfilingSection("name") to add events and popProfilingSection() to stop the last event.
These events are marked as fundamental and scoped under solver.initialize/ or solver.advance/.

Motivation and additional information

This should simplify the integration of profiling for adapter and tools into the precice profiling.

Closes #1647

Author's checklist

  • I used the pre-commit hook to prevent dirty commits and used pre-commit run --all to format old commits.
  • I added a changelog file with make changelog if there are user-observable changes since the last release.
  • I added a test to cover the proposed changes in our test suite.
  • For breaking changes: I documented the changes in the appropriate porting guide.
  • I sticked to C++17 features.
  • I sticked to CMake version 3.16.3.
  • I squashed / am about to squash all commits that should be seen as one.

Reviewers' checklist

  • Does the changelog entry make sense? Is it formatted correctly?
  • Do you understand the code changes?

@fsimonis fsimonis added the enhancement A new feature, a new functionality of preCICE (from user perspective) label May 12, 2023
@uekerman
Copy link
Member

Is it clear for the user that with sync-mode="on", they could create their own deadlock?

In general, I am a bit afraid that we are opening another box for v3.0 🙈

@fsimonis
Copy link
Member Author

I don't think that user events should be synchronised and in this PR they aren't. This can always be done in user code.

@davidscn
Copy link
Member

Right now, I don't see the real use-case for these API functions. Main reasons:

  • the adapters speak the language of the solvers and if the solver is performance oriented, they will have their own performance instrumentation, which is also more natural to use then
  • for tools like ASTE, the interesting part happens in preCICE, not in ASTE
  • we already measure the solver.advance calls which are a summary of the solver events within a time-step

To me this API sound more like an extension of the framework we want to couple.

@fsimonis
Copy link
Member Author

fsimonis commented May 15, 2023

the adapters speak the language of the solvers and if the solver is performance oriented, they will have their own performance instrumentation, which is also more natural to use then

Yes in theory.
In practise, this doesn't seem to be straight forward though (FEniCS, OpenFOAM). Maybe we can get some input by other adapter devs here: @BenjaminRodenberg @IshaanDesai @MakisH @uekerman

A valuable use-case in my mind is allowing to profile adapters of commercial and/or closed-source codes such as Fluent @mtree22 or TAU.
Some tools like CalculiX also don't use profiling themselves.

for tools like ASTE, the interesting part happens in preCICE, not in ASTE

True.

However, it would remove the need for custom profiling inside ASTE.
Given that ASTE is basically pure IO and distributed filesystems tend to be slow, it may be interesting to profile the time spend creating output files for example. Especially as we use ASTE on big distributed machines.

Other tools could profit from this functionality, prime example being the micro-manager @IshaanDesai.

Hopefully I tagged enough people to get some input 😉

@uekerman
Copy link
Member

IMO, this is useful, but I would postpone it to 3.x.

@IshaanDesai
Copy link
Member

This API would be quite useful in the Micro Manager. But the names of the functions confused me a bit, wouldn't startProfiling("name") and stopProfiling() make more sense?

@fsimonis fsimonis force-pushed the add-profiling-api branch from 8e6801c to 03487f0 Compare May 15, 2023 12:57
@fsimonis fsimonis added this to the Version 3.x.x milestone May 17, 2023
@fsimonis
Copy link
Member Author

fsimonis commented Jun 2, 2023

This will also work nicely with #1673

@fsimonis
Copy link
Member Author

fsimonis commented Aug 13, 2024

@IshaanDesai are you using this API? Would you support introducing this feature into preCICE?

@MakisH how do you see the value of such an API for analysing the performance of adapter code?

@MakisH
Copy link
Member

MakisH commented Aug 22, 2024

@MakisH how do you see the value of such an API for analysing the performance of adapter code?

On the one hand, I feel we risk getting into foreign territories by integrating such features into preCICE itself (profiling, logging, potentially more).

On the other hand, the nice aspect in both is not so much the "we already can do it, so why not", but the fact that the adapter and preCICE events/logs are in the same place and format, so they can be analyzed in context. Hence, it also fits the standardization context.

In the OpenFOAM adapter, we already implemented custom profiling for the reference paper, which would have been much easier if this feature existed already: precice/openfoam-adapter#256 Because the adapter now has profiling, there is no pressing need for the preCICE profiling at the moment.

Still, I guess this could be mildly interesting in case a user comes with their own code and claim that they have performance issues. We can already distinguish between the time in preCICE and outside preCICE, but we could then also point them to more specific directions to improve the runtime of their code. But I don't think this is our job at the moment, and profiling mainly makes sense for us in the context of evaluating and publishing our adapters.

I would see this as a "nice to have if it does not introduce any debt, but low priority".

@fsimonis
Copy link
Member Author

On the other hand, the nice aspect in both is not so much the "we already can do it, so why not", but the fact that the adapter and preCICE events/logs are in the same place and format, so they can be analyzed in context. Hence, it also fits the standardization context.

The profiling and logging in the adapter always boils down to the question:

Does the solver already provide this functionality in an accessible way?

If the answer is no, then such an API is very useful.
If the answer is yes, then it is most natural for the end user to use the solver solution.
This makes standardization difficult though.

In the OpenFOAM adapter, we already implemented custom profiling for the reference paper, which would have been much easier if this feature existed already: precice/openfoam-adapter#256 Because the adapter now has profiling, there is no pressing need for the preCICE profiling at the moment.

The upside of the preCICE profiling is that it is robust to failures, works for parallel solvers even outside MPI environments, and comes with tooling.

Have you tried the OpenFOAM timers when running with mpi?

@MakisH
Copy link
Member

MakisH commented Aug 22, 2024

The profiling and logging in the adapter always boils down to the question:

Does the solver already provide this functionality in an accessible way?

If the answer is no, then such an API is very useful.

Yes, but provided that profiling itself is indeed useful (for more people than just us).

Have you tried the OpenFOAM timers when running with mpi?

These custom timers do not do anything special to account for discrepancies between ranks. We just measure the main rank.

@IshaanDesai
Copy link
Member

are you using this API? Would you support introducing this feature into preCICE?

Currently not using this API, and right now I do not see an urgent use case. I am currently trying to profile functionality in the Micro Manager, and not functionality related to coupling. For multiscale scenarios, the coupling itself is one of the cheapest operations, and hence not the focus of profiling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A new feature, a new functionality of preCICE (from user perspective)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Profiling of user-code via API
5 participants