Closed
Description
Hi,
I just installed 1.0.1 on MacOSx. I find that the model is assigning zero probability to oov words:
import spacy
nlp=spacy.load('en')
x=nlp(u'this is an oovword')
[(t, t.is_oov, t.prob) for t in x]
[(this, False, -5.36181640625), (is, False, -4.457748889923096), (an, False, -6.014852046966553), (oovword, True, 0.0)]
More context: I had the same experience as the issue referenced here #535, and so I did these sequence of steps:
uninstall 0.100.0, install 1.0.1, download data, uninstall 1.0.1, install 1.0.1, download data