Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: signal: Compute envelope of a real- or complex-valued signal #20097

Merged
merged 18 commits into from
Dec 2, 2024

Conversation

DietBru
Copy link
Contributor

@DietBru DietBru commented Feb 16, 2024

This PR proposes a new function for scipy.signal, which computes the envelope of a real- or complex-valued signal after applying an optional bandpass filter.

The discussion in #19814 revealed that literature is a bit convoluted regarding on how to define envelopes for signals. Hence, this PR also tries to concisely document the approach utilized here.

This PR differs from #19814 in that it provides a general solution, whereas #19814 does not produce correct results (regarding the definitions of a envelope given here) for all possible signals.

@github-actions github-actions bot added scipy.signal enhancement A new feature or improvement labels Feb 16, 2024
@DietBru
Copy link
Contributor Author

DietBru commented Feb 16, 2024

The rendered version can be found here.

I am not sure what to make of the error message:

scipy/signal/_signaltools.py:2467: in <module>
    def envelope(z: np.ndarray, bp_in: tuple[int | None, int | None] = (1, None), *,
E   TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

This is the only error I found in the failing tests which could be related this PR.

@lucascolley
Copy link
Member

lucascolley commented Feb 16, 2024

That syntax is not supported in Python < 3.10. from __future__ import annotations at the top of the file might work. Keeping the typing in a stub .pyi file may be easier to work with for now. Alternatively, you can use Union, however, I think the linter may complain about that (not sure).

@DietBru
Copy link
Contributor Author

DietBru commented Feb 17, 2024

Thanks @lucascolley, for the prompt answer. It could have been that I asked this question before 😅
Unfortunately, python dev.py lint does not complain...

@j-bowhay
Copy link
Member

j-bowhay commented Feb 17, 2024

@yagizolmez would you be interested in reviewing this given your heavy involvement in previous discussions?

@yagizolmez
Copy link
Contributor

@j-bowhay I will leave a review in the upcoming days. I want to note that this is NOT a competing PR with #19814. Following the discussion we had there, I decided to break up the envelope function I proposed and not implement the analytical envelope.

@DietBru I have a quick question before I leave a more detailed review. Comparing this function with hilbert, major additions seem to be:

  1. Support for complex-valued signals.
  2. The option to upsample the envelope.
  3. The option to apply a bandpass filter.

Have you considered extending hilbert rather than adding a new function?

@DietBru
Copy link
Contributor Author

DietBru commented Feb 18, 2024

Have you considered extending hilbert rather than adding a new function?

I did, but I came to the conclusion that that would only lead to confusion:

The analytic signal is a concept based on real-valued signals. Rightfully, hilbert() raises a ValueError for complex-valued inputs. Also, changing that behavior could break existing implementations.

The envelope on the other hand is a concept that is inherent to complex-valued signals with real-valued signals being a special case. Note that the treatment of real-valued signals depends on the underlying assumptions of what the signal represents. This is illustrated in the second example of the envelope()'s doc string.

@yagizolmez
Copy link
Contributor

@DietBru I am sorry for the delay. I have been very busy these days, so I am not yet able to provide a detailed review. One thing catched my eye. Under certain circumstances, this function must give the same result with hilbert(), i.e. when z is real and bp_in is selected appropriately. Have you considered writing some tests and benchmarks covering these situations?

@DietBru
Copy link
Contributor Author

DietBru commented Feb 26, 2024

One thing catched my eye. Under certain circumstances, this function must give the same result with hilbert(), ...

Good catch, thank you. I found an edge case where the results differ. Hence, I did some refactoring to ensure that envelope() and hilbert() always behave identically. I think it should ready for complete review now.

@DietBru DietBru added this to the 1.14.0 milestone Apr 9, 2024
@lucascolley
Copy link
Member

@yagizolmez would you like to take another look at this?

@yagizolmez
Copy link
Contributor

@lucascolley Thanks for reminding me Lucas. I have gotten extremely busy in the school lately. I will take a look when I have time. @DietBru I apologize.

@lucascolley lucascolley changed the title ENH: Compute envelope of a real- or complex-valued signal. ENH: signal: Compute envelope of a real- or complex-valued signal May 18, 2024
@DietBru
Copy link
Contributor Author

DietBru commented May 27, 2024

Resolved merge conflict. Current version of the rendered documentation can be found here.
Test failures seem unrelated.

Copy link
Contributor

@tylerjereddy tylerjereddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read through the discussion here.

This looks like a decent chunk of work, but I'll have to bump the release milestone since there's no detailed expert review of the new feature. I could check it for superficial things like test coverage and so on, but I wouldn't feel comfortable merging it on my own. Apologies!

scipy/signal/tests/test_signaltools.py Outdated Show resolved Hide resolved
@tylerjereddy tylerjereddy modified the milestones: 1.14.0, 1.15.0 May 27, 2024
Copy link
Contributor

@yagizolmez yagizolmez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DietBru Sorry for the delay. In order to keep things moving, I reviewed the documentation part of the code (which may eventually cause changes to the actual code). I haven't reviewed the examples yet, because the docs don't render. I will review the other parts soon.

scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
Comment on lines 2468 to 2467
Any complex-valued signal `z(t)` can be split into a real-valued (instantaneous)
amplitude `a(t)` and a real-valued (instantaneous) phase `phi(t)`, i.e.,
`z(t) = a(t) * exp(1j*phi(t))`. The envelope is defined as the absolute value of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Any complex-valued signal `z(t)` can be split into a real-valued (instantaneous)
amplitude `a(t)` and a real-valued (instantaneous) phase `phi(t)`, i.e.,
`z(t) = a(t) * exp(1j*phi(t))`. The envelope is defined as the absolute value of
Any complex-valued signal `z(t)` can be described by a real-valued amplitude
`a(t)` and a real-valued phase `phi(t)`, i.e., `z(t) = a(t) * exp(1j*phi(t))`. The
envelope is defined as the absolute value of

The word 'instantaneous' is redundant here, since you are already describing the amplitude and phase as a function of time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The redundancy is intentional here. In signal processing, when talking about phase and amplitude it is often implicitly assumed that those are stationary. I think, the redundancy makes it easier to grasp for many people.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the wording slightly in commit f910d6e

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to write phi in Greek letters. Could you write the math expressions in latex? If this is not possible, I would suggest denoting the phase with p instead of phi.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also like the LaTeX expression better, but not all IDEs render those correctly when displaying help texts.

There is guidance on that topic in SciPy, though not very straightforward to find:
The SciPy contributor Guide links to the Numpy Documentation Style , which links to Numpydoc guide
which states that LaTeX may be used in the Notes section.

So I followed that convention.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess using p(t) instead of phi(t) is a matter of taste—I'm just used to signal processing literature where an phase angle is always denoted by a Greek letter.

scipy/signal/_signaltools.py Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Show resolved Hide resolved
@DietBru
Copy link
Contributor Author

DietBru commented Jul 19, 2024

Thanks for the review @yagizolmez. The updated rendered version of the documentation can be found here,

Copy link
Contributor

@yagizolmez yagizolmez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DietBru I did a second pass on the documentation, now that the examples are rendered. Also, I would like to point out one thing. I don't have much experience ever needing to extract the envelope of a complex-valued signal. I can review for math accuracy, but I cannot review for user-friendliness/usefulness etc.

@ilayn Do you know anyone who could provide a better review for the complex-valued signal case?

scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
@lucascolley
Copy link
Member

Do you know anyone who could provide a better review for the complex-valued signal case?

Perhaps @roryyorke or @larsoner ?

Copy link
Contributor

@yagizolmez yagizolmez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation largely looks good to me. Although I didn't understand some parts. I will start going over the unit tests.

scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Show resolved Hide resolved
Copy link
Contributor

@yagizolmez yagizolmez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass at the tests looks good. I will take a second look in the near future. I think you could add some tests for squared = False too? This could also be calculated by hand for simple signals.

scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/tests/test_signaltools.py Show resolved Hide resolved
scipy/signal/tests/test_signaltools.py Show resolved Hide resolved
scipy/signal/tests/test_signaltools.py Show resolved Hide resolved
scipy/signal/tests/test_signaltools.py Show resolved Hide resolved
@DietBru
Copy link
Contributor Author

DietBru commented Aug 7, 2024

The failure in building the docs seem to be a jupyterlite_sphinx problem, hence, unreleated to this PR—will look into that later on.

DietBru and others added 3 commits August 11, 2024 13:11
Remove unrelated whitespaces

Co-authored-by: Yagiz Olmez <yagiz.olmez@gmail.com>
Co-authored-by: Yagiz Olmez <yagiz.olmez@gmail.com>
@DietBru
Copy link
Contributor Author

DietBru commented Aug 11, 2024

Rebased this branch on the documentation is building again. The current envelope documentation can be found here. The test failures are generated by special/tests/test_orthogonal_eval.py, hence unrelated to this PR.

Thanks again for your input @yagizolmez — I hope I have sufficiently addressed all your remarks by now.

@yagizolmez
Copy link
Contributor

@DietBru Thanks. I will check in a few days.

Copy link
Contributor

@yagizolmez yagizolmez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! I would suggest getting at least one more opinion before merging though.

scipy/signal/tests/test_signaltools.py Show resolved Hide resolved
@j-bowhay j-bowhay added the needs-release-note a maintainer should add a release note written by a reviewer/author to the wiki label Oct 8, 2024
Copy link
Member

@j-bowhay j-bowhay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @DietBru a few suggestion but otherwise looks in good shape

scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/tests/test_signaltools.py Outdated Show resolved Hide resolved
@DietBru
Copy link
Contributor Author

DietBru commented Oct 15, 2024

Thanks for the feedback @j-bowhay! I think I have incorporated all your suggestions—the rendered documentation can be found here.

Co-authored-by: Jake Bowhay <60778417+j-bowhay@users.noreply.github.com>
Copy link
Member

@lucascolley lucascolley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @DietBru!! Given @yagizolmez's approval, I'm happy to merge this, modulo the suggestions I have added here, and checking the rendered docs one more time. Let's get this in for 1.15.0 🎉

Please check all of my suggestions and commit them if you agree.

scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/_signaltools.py Outdated Show resolved Hide resolved
scipy/signal/tests/test_signaltools.py Outdated Show resolved Hide resolved
DietBru and others added 2 commits December 1, 2024 09:09
Fixed typo

Co-authored-by: Lucas Colley <lucas.colley8@gmail.com>
* In class `TestEnvelope.test_envelope_invalid_parameters()` make linter statement
  `# noinspection PyTypeChecker` expression-specific instead of method-specific
@DietBru
Copy link
Contributor Author

DietBru commented Dec 2, 2024

Thanks @lucascolley for the review and spotting the typos—I think, I addressed all your comments in the last commit.
The CI errors seem to be caused by #21988 and thus unrelated. From my side it is ready for merging.

@lucascolley
Copy link
Member

thanks Dietrich, let's check the rendered docs once gh-21988 is resolved.

@DietBru
Copy link
Contributor Author

DietBru commented Dec 2, 2024

thanks Dietrich, let's check the rendered docs once gh-21988 is resolved.

Glancing at the rendered docs, I saw no surprises. 🎉

@lucascolley lucascolley merged commit 876c84d into scipy:main Dec 2, 2024
@DietBru DietBru deleted the EnvelopeHIlbert branch January 19, 2025 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A new feature or improvement needs-release-note a maintainer should add a release note written by a reviewer/author to the wiki scipy.signal
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants