Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistency doc / code: predict_log_density #2121

Open
AthenaisGautier opened this issue Oct 30, 2024 · 2 comments
Open

Inconsistency doc / code: predict_log_density #2121

AthenaisGautier opened this issue Oct 30, 2024 · 2 comments

Comments

@AthenaisGautier
Copy link

Documentation/tutorial notebooks

When comparing the documentation of "predict_log_density" to source code, I found the following statement:

 Given a Normal distribution for the latent function, and a datum Y, compute the log predictive density of Y,

    i.e. if
        q(F) = N(Fmu, Fvar)

    and this object represents

        p(y|F)

    then this method computes the predictive density

        log ∫ p(y=Y|F)q(F) df

    :param X: input tensor
    :param Fmu: mean function evaluation tensor
    :param Fvar: variance of function evaluation tensor
    :param Y: observation tensor
    :returns: log predictive density

while the code for predict_log_density in the Gaussian class is:

    def _predict_log_density(
      self, X: TensorType, Fmu: TensorType, Fvar: TensorType, Y: TensorType
    ) -> tf.Tensor:
      return tf.reduce_sum(logdensities.gaussian(Y, Fmu, Fvar + self._variance(X)), axis=-1)

which, as far as I understand, corresponds to ∫ log p(y=Y|F)q(F) df (i.e. expected log-density) ?

Is this a lack of consistency between the documentation and code, or am I just l misunderstanding ?
Thanks and cheers!

@uri-granta
Copy link
Contributor

Sorry for the delayed response! Good spot: it certainly looks like we're returning the predictive log density rather than the log predictive density as claimed in the docstring. I will check with someone who understands this better than me and post an update...

@uri-granta
Copy link
Contributor

Ok, my understanding is that the docs are correct but perhaps incomplete. The code is calculating the log predictive density for each output and summing them (equivalent to the the log of the product). The integral alluded to in the docs is actually inside logdensities.gaussian. I think this corresponds to equation 2.9 in https://gaussianprocess.org/gpml/chapters/RW2.pdf. The calculation itself certainly seems to corresponds to equation 2.34 (though positive rather than negative).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants