Hi Greg. I took another look at this issue. As mentioned here: #20 (comment) a sig_ml.clip of 0.25 may not be sensible.

Thinking about the problem further, this error occurs when corex assigns a selection of datapoints to a given n_hidden by dim_hidden combination such that each of the datapoints has the identical value - this is required for sig_ml to be calculated =0 in estimate_parameters. This also means that the marginals for those datapoints =0, and therefore the log(marginal) is undefined, since log(0) = -inf.

In the code this happens in the marginal_p function line:
return (-(xi - mu)**2 / (2. * sig) - 0.5 * np.log(2 * np.pi * sig)).transpose((1, 0, 2)) # log p(xi|yj)
where sig = 0 causes a divide-by-zero in the first term, and a log(0) in the second term.

However, thinking about this for a while, it seems to me that while leads a computational error, a selection of datapoints with the same value is not a gaussian distribution and so the real problem is that the selected data does not fit the model. Furthermore, this is not likely to happen with actually continuous gaussian data, since its unlikely that multiple datapoints would have the same value, however is much more likely to happen when binary data is used with the 'gaussian' marginal_distribution as in the example above, or if for example a categorial variable is coded as 0, 1 alongside truly continuous/gaussian data.

So, perhaps the solution here is not to clip sig at all (since I'm not sure one can define a sig.clip that would be valid for all datasets), but to transfer or exclude problematic variables in the data?