plot3d crashes on floats with many decimal places

[dietmar]

mayavi 4.1.0 on Debian wheezy amd64

When I call plot3d with 6 floats that have many decimal places, mayavi crashes with "TypeError: argument 1: value is "

These values produce the crash for me:

xs = [-8.3747958651808272, -8.1719663022434617]
ys = [107.00242750501148, 103.32517068758831]
zs = [39.928090191498192, 41.552475382193073]

If I round them to fewer decimal places, everything works fine:

xs = [ round(a * 1e5) / 1e5 for a in xs ]
ys = [ round(a * 1e5) / 1e5 for a in ys ]
zs = [ round(a * 1e5) / 1e5 for a in zs ]

Traceback:

$ python
Python 2.7.3 (default, Jan  2 2013, 13:56:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from mayavi import mlab
>>> xs = [-8.3747958651808272, -8.1719663022434617]
>>> ys = [107.00242750501148, 103.32517068758831]
>>> zs = [39.928090191498192, 41.552475382193073]
>>> mlab.plot3d(xs, ys, zs)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/mayavi/tools/helper_functions.py", line 34, in the_function
    return pipeline(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/mayavi/tools/helper_functions.py", line 79, in __call__
    output = self.__call_internal__(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/mayavi/tools/helper_functions.py", line 536, in __call_internal__
    self.source = self._source_function(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/mayavi/tools/sources.py", line 1171, in line_source
    data_source.reset(x=x, y=y, z=z, scalars=s)
  File "/usr/lib/python2.7/dist-packages/mayavi/tools/sources.py", line 508, in reset
    pd.set(points=points)
  File "/usr/lib/python2.7/dist-packages/traits/has_traits.py", line 1952, in trait_set
    setattr( self, name, value )
  File "/usr/lib/python2.7/dist-packages/tvtk/tvtk_classes.zip/tvtk_classes/point_set.py", line 41, in _set_points
  File "/usr/lib/python2.7/dist-packages/tvtk/array_handler.py", line 752, in deref_array
    ret.append(convert_array(a, s))
  File "/usr/lib/python2.7/dist-packages/tvtk/array_handler.py", line 633, in convert_array
    return conv[vtk_typ](arr, vtk_arr)
  File "/usr/lib/python2.7/dist-packages/tvtk/array_handler.py", line 569, in array2vtkPoints
    vtk_array = array2vtk(arr)
  File "/usr/lib/python2.7/dist-packages/tvtk/array_handler.py", line 298, in array2vtk
    result_array.SetVoidArray(numpy.getbuffer(z_flat), len(z_flat), 1)
TypeError: argument 1: value is 

[prabhuramachandran]

Thanks for the reproducible example. I confirm that this happens for me as well and will take a look when I have some time. The problem also does not occur if you have 3 points, say if you add 1 to the end of each of these lists...

[stefanoborini]

Confirmed also on OSX in the latest mayavi with VTK 6.3

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/sborini/Work/github/mayavi/mayavi/tools/helper_functions.py", line 37, in the_function
    return pipeline(*args, **kwargs)
  File "/Users/sborini/Work/github/mayavi/mayavi/tools/helper_functions.py", line 82, in __call__
    output = self.__call_internal__(*args, **kwargs)
  File "/Users/sborini/Work/github/mayavi/mayavi/tools/helper_functions.py", line 523, in __call_internal__
    self.source = self._source_function(*args, **kwargs)
  File "/Users/sborini/Work/github/mayavi/mayavi/tools/sources.py", line 1174, in line_source
    data_source.reset(x=x, y=y, z=z, scalars=s)
  File "/Users/sborini/Work/github/mayavi/mayavi/tools/sources.py", line 488, in reset
    pd.set(points=points)
  File "/Users/sborini/.jaguar/envs/jaguar/lib/python2.7/site-packages/traits/has_traits.py", line 1541, in trait_set
    setattr( self, name, value )
  File "tvtk_classes/point_set.py", line 54, in _set_points
  File "/Users/sborini/Work/github/mayavi/tvtk/array_handler.py", line 772, in deref_array
    ret.append(convert_array(a, s))
  File "/Users/sborini/Work/github/mayavi/tvtk/array_handler.py", line 653, in convert_array
    return conv[vtk_typ](arr, vtk_arr)
  File "/Users/sborini/Work/github/mayavi/tvtk/array_handler.py", line 589, in array2vtkPoints
    vtk_array = array2vtk(arr)
  File "/Users/sborini/Work/github/mayavi/tvtk/array_handler.py", line 304, in array2vtk
    result_array.SetVoidArray(getbuffer(z_flat), len(z_flat), 1)
TypeError: SetVoidArray argument 1: value is _A_>? ?????'?Z@?Ψ??C@?L??
                                                                      X ??3????Y@?.i???D@, required type is p_void

[stefanoborini]

Ok, this is a funny one...

The problem is the following: the core issue is the routine array2vtk, where we set the content of our numpy array to the vtk object.

result_array.SetVoidArray(getbuffer(z_flat), len(z_flat), 1)

getbuffer is a shorthand for numpy.getbuffer, which returns the raw bytes of the numpy array. Sounds all fine until...

until we examine the content for the particular numerical example. See the '_A_' part? that is just due to the content of the first array

>>> str(np.getbuffer(np.array([-8.3747958651808272])))
'_A_>\xe5\xbf \xc0'
>>> str(np.getbuffer(np.array([-8.3747958])))
'7[/<\xe5\xbf \xc0'

Unfortunately, vtk has a different opinion when it comes to that number. If you look in Wrapping/PythonCore/vtkPythonUtil.cxx there's the core of the problem. due to that specific byte content _A_, the routine vtkPythonUtil::UnmanglePointer(char *ptrText, int *len, const char *type)
believes the pointer is some SWIG like pointer, and tries to apply logic according to this belief

 999     // If no null bytes, then do a full check for a swig pointer                                                                                      
1000     if (i == 0)                                                                                                                                       
1001       {                                                                                                                                               
1002       unsigned long long l;                                                                                                                           
1003       i = sscanf(text, "_%llx_%s", &l ,typeCheck);                                                                                                    
1004       u.l = static_cast<uintptr_t>(l);                    

and the thing crashes.

How to solve? It should try to demangle, and if it fails, consider it a pure array (and not a swig array), instead of assuming it is, and fail to match the type.

But in any case, it's a VTK bug, it still exists in vtk7, and I will post it in the VTK bug tracker to hear their opinion.

From our side, the only possible workarounds are:

• always convert it to a swig array, or
• don't use any floating point value whose representation contains X with X any char...

both not very appropriate I'd say. I'll try to find a more appropriate one.

[stefanoborini]

It is logically not possible to workaround this bug on vtk 6.3. The only possible way would be to upgrade to vtk7 and use memoryview exclusively in our code, instead of getbuffer, so that we are guaranteed a format and that format is 'd'. vtk6.3 always interpret it via the buffer protocol no matter what, with the consequence that any data that looks enough like a swig pointer converted as a string will be parsed as such by vtk. @prabhuramachandran do you have any opinion on this?

[stefanoborini]

Posted in vtk bugtracker as http://www.vtk.org/Bug/view.php?id=16054

[stefanoborini]

VTK fix at https://gitlab.kitware.com/vtk/vtk/merge_requests/1469

[stefanoborini]

Fix coming for 7.1, but not merged yet in 6.3

http://www.vtk.org/Bug/view.php?id=16054#c36080