Open
Description
... is currently broken:
Traceback (most recent call last):
File "/data/workspace/linux/JyNI/JyNI-Demo/src/JyNINumPyTest.py", line 68, in <module>
import numpy as np
File "/data/workspace/linux/numpy/1.14.0/numpy/__init__.py", line 168, in <module>
from . import ma
File "/data/workspace/linux/numpy/1.14.0/numpy/ma/__init__.py", line 44, in <module>
from . import core
File "/data/workspace/linux/numpy/1.14.0/numpy/ma/core.py", line 6333, in <module>
masked = masked_singleton = MaskedConstant()
File "/data/workspace/linux/numpy/1.14.0/numpy/ma/core.py", line 6270, in __new__
cls.__singleton = MaskedArray(data, mask=mask).view(cls)
File "/data/workspace/linux/numpy/1.14.0/numpy/ma/core.py", line 2790, in __new__
_data = ndarray.view(_data, cls)
AttributeError: 'type$1' object has no attribute 'update'
Stick to NumPy 13.3 until this is solved...
Metadata
Assignees
Labels
No labels
Activity
Stewori commentedon Jun 30, 2018
I just investigated newer NumPy versions. 14.1 and 14.2 result in the same error as 14.0 above.
NumPy 14.3, 14.4, 14.5 give this new error:
Stewori commentedon Jul 1, 2018
So, I located the first type of failure to NumPy's source file
multiarray/ctors.c
.Specifically in the function
PyArray_NewFromDescr_int
the lineres = PyObject_Call(func, args, NULL);
fails, i.e. returns zero, triggering the subsequentgoto fail;
. Next step is to identify why this call fails. For now I only know that it emits a bunch of Python-exceptions, where the message that we see, i.e.'type$1' object has no attribute 'update'
is only the tip of the iceberg. The full sequence of error messages is:So far I conclude that somehow attribute (write-)access of
numpy.ndarray
got messed up.Stewori commentedon Jul 2, 2018
Tracked it down a bit further:
It fails in the method
MaskedArray._update_from
inma/core.py
.Specifically the line
self.__dict__.update(_dict)
fails. Here_dict
contains a bunch of updates for attributes that shall be written toself.__dict__
. It seems that for some reasonself.__dict__
might not be an ordinary dictionary, which somehow causes the issue.type(self.__dict__)
results in<type 'type$1'>
, which is familiar from our error message'type$1' object has no attribute 'update'
.print(self.__dict__)
outputs<attribute '__dict__' of 'MaskedArray' objects>
. So maybe it's an issue with attribute wrappers while the dictionary itself is fine. It seems like the key lies in understanding whattype$1
actually is.The
MaskedArray
class extendsndarray
and I remember that I once observed Astropy to fail with JyNI because ofndarray
subclassing. Maybe before version 14.0 the NumPy import worked without subclassingndarray
. Unfortunately the commit history ofma/core.py
is insanely complex and it is challenging to track down the breaking commit. I thought about using git's bisect feature, but that's probably tedious because of all the rebuilds required. For now I will just continue investigation. If I can fixndarray
subclassing in JyNI, then it hopefully also improves usecases like in Astropy.Stewori commentedon Jul 4, 2018
Note that the CPython output is as one would naively expect:
Instead of
<type 'type$1'>
we get<type 'dict'>
and instead of<attribute '__dict__' of 'MaskedArray' objects>
we get{}
. So somehow JyNI, or maybe Jython already, messes up this attribute. Maybe it just misses to evaluate the attribute wrapper properly.Stewori commentedon Jul 5, 2018
The "dict" object of type
<type 'type$1'>
is an instance ofPyDataDescriptor
, seePyDataDescriptor.java
.type$1
is a generated type (as I already expected actually).PyDataDescriptor
is abstract and Jython generates concrete implementations at runtime, resulting in these strange type names. Now the real question is why JyNI does not evaluate__get__
. Maybe the case of extending natively defined builtin-like types is not yet properly covered, an observation that is consistent with the way how Astropy fails.I will review the way how new-style classes are handled by JyNI. Originally the described case should have been workable, but that appears to be incomplete. Maybe this will require something like
PyCPeerDerived
. We'll see...Stewori commentedon Jul 5, 2018
Quick experiment:
Inserted this code snippet into
PyCPeer.__findattr_ex__
right before the return statement:Now the descriptor is evaluated, but so far this doesn't fix anything because it just evaluates to
None
. It results now inAttributeError: 'NoneType' object has no attribute 'update'
instead ofAttributeError: 'type$1' object has no attribute 'update'
(Is this an improvement?).It seems like the generated implementation of
PyDataDescriptor
does not know what to do with aPyCPeer
...Stewori commentedon Jul 26, 2018
The recently released NumPy 1.15.0 gives the following error on import:
However, I suppose this is easier to fix. Adding a
PyStructSequence_InitType
implementation is not a big deal. However I suspect there will be subsequent issues.ShaharZivan commentedon Nov 27, 2018
Did you find any solution for this yet? Even a temporary hardcoded patch would do. I need to use sequitur-g2p with Jython and sadly it requires NumPy 1.14.2 or higher.