@emanuele-moscato I've left around several todo markers for points to discuss. You can find them with #TODO GA

Look at the explainers under ferret/explainers/explanation_speech/:

• Whenever the path to an audio file is required (typically in the compute_explanation methods), pass an object of type FerretAudio (newly defined in the PR).
• Make FerretAudio able to transcribe to text (dedicated method called from within the explainers when/if needed). --> NOT VALID ANYMORE
• Check the SpeechXAI examples notebook.

To do:

• In the definition of the ExplanationSpeech class, change the kwarg name from audio_path to audio and fix code accordingly everywhere the class is implemented (make sure an object of type FerretAudio is passed).
• Address the comments in the last code review.
• Complete the update of the speech explainers so they accept an object of type FerretAudio as their input.
• See previous to-do list.

Note: the audio transcription functions transcribe_audio and transcribe_audio_with_model were moved from the ferret/explainers/explanation_speech/utils_removal.py module to ferret/speechxai_utils.py to avoid a circular import. The rest of the code has already been updated, but have one last check please!

Issues to solve:

• When loading an audio file into a FerretAudio object and then extracting the AudioSegment object (to_pydub method), the audio gets distorted. Check FerretAudio's to_pydub method.

Action: modified FerretAudio's to_pydub method so as to create a pudyb AudioSegment object from an array that is always unnormalized and of dtype int16.

• Move audio resampling (if the native sample rate is different from 16 KHz) to within the transcription method.
• Move normalization of the array corresponding to the audio (if needed) to within the transcription method as well (so there are no ambiguities on whether the array provided by the user is normalized or not: it is what it is from the start).
• Check that the new way of obtaining arrays from audio (librosa.load) returns the same as (AudioSegment.load_wav --> pydub_to_np).

Result: numerically it is, they both return an array of dtype float32 normalized by a factor 2 ** 15, but the shape is different as (for mono audio) librosa returns a flattened array while pydub_to_np returns an array of shape (n_samples, 1).

• Raise error if the numpy array or audio file passed to FerretAudio has more than one channel (we only support mono audio!).

Action: this is inferred looking at the shape of the array. 1-dimensional array --> good, 2-dimensional array --> good only if the trailing dimension is 1 (shape (n_samples, 1)), array of dimension > 2 --> shape not understood, raise exception.

• Remove the remove_word_np function if it ends up not being used.

Regarding the normalization: we should not touch the input array unless 1) it's not normalized and 2) we are transcribing it with whisperX -- that's the only case where we really need normalization + 16kHz sampling rate.