arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Micah Kornfield <emkornfi...@gmail.com>
Subject Re: [Cython] Pyarrow Cython Crash
Date Sat, 15 Aug 2020 04:43:08 GMT
Hi Shawn,
doing a web search seems to show that you might be two versions of
libstdc++ in the program.  Did you compile pyarrow from source?

Thanks,
Micah

On Tue, Aug 11, 2020 at 3:50 AM Shawn Yang <shawn.ck.yang@gmail.com> wrote:

> Hi, I'm using pyarrow cython api on macos.
> 1. OS info:
> ProductName: Mac OS X
> ProductVersion: 10.13.6
> BuildVersion: 17G2112
> 2. Clang info:
> Configured with: --prefix=/Library/Developer/CommandLineTools/usr
> --with-gxx-include-dir=/usr/include/c++/4.2.1
> Apple LLVM version 10.0.0 (clang-1000.10.44.4)
> Target: x86_64-apple-darwin17.7.0
> Thread model: posix
> InstalledDir: /Library/Developer/CommandLineTools/usr/bin
>
> 3. Code
> setup.py:
>
> from distutils.core import setup
> from Cython.Build import cythonize
>
> import os
> import numpy as np
> import pyarrow as pa
>
>
> ext_modules = cythonize("example.pyx")
>
> for ext in ext_modules:
>     # The Numpy C headers are currently required
>     ext.include_dirs.append(np.get_include())
>     ext.include_dirs.append(pa.get_include())
>     ext.libraries.extend(pa.get_libraries())
>     ext.library_dirs.extend(pa.get_library_dirs())
>
>     if os.name == 'posix':
>         ext.extra_compile_args.append('-std=c++11')
>         # ext.extra_compile_args.append('-stdlib=libc++')
>
>     # Try uncommenting the following line on Linux
>     # if you get weird linker errors or runtime crashes
>     ext.define_macros.append(("_GLIBCXX_USE_CXX11_ABI", "0"))
>
>
> setup(ext_modules=ext_modules)
>
> example.pyx:
>
> # distutils: language=c++
>
> from pyarrow.lib cimport *
> from libcpp.memory cimport shared_ptr, make_shared, dynamic_pointer_cast
> from libc.stdint cimport *
> from cpython cimport *
> from pyarrow.lib cimport Schema, DataType, ListType, MapType, StructType, Field
> from pyarrow.lib cimport CSchema, CField, CDataType, CListType, CMapType
>
> from pyarrow cimport import_pyarrow
>
> import_pyarrow()
>
>
> def get_array_length(obj):
>     # Just an example function accessing both the pyarrow Cython API
>     # and the Arrow C++ API
>     cdef shared_ptr[CArray] arr = pyarrow_unwrap_array(obj)
>     if arr.get() == NULL:
>         raise TypeError("not an array")
>     return arr.get().length()
>
> def cast_type():
>     import pyarrow as pa
>     t = pa.list_(pa.string())
>     cdef:
>         shared_ptr[CDataType] c_type = pyarrow_unwrap_data_type(t)
>         shared_ptr[CListType] c_list_type = dynamic_pointer_cast[CListType, CDataType](c_type)
>
>
>
> 4. Error
> When I execute `python setup.py build_ext --inplace`, I got
> example.cpp:649:10: fatal error: 'unordered_map' file not found #include
> <unordered_map>.
> When I add `-stdlib=libc++`, This error is gone. But when I execute `
> python -c "import example;example.cast_type()"`, it crashed. The stack is :
>   * frame #0: 0x00007fff67cc2c9a
> libc++abi.dylib`__cxxabiv1::__si_class_type_info::has_unambiguous_public_base(__cxxabiv1::__dynamic_cast_info*,
> void*, int) const
>     frame #1: 0x00000001053d69c8 libstdc++.6.dylib`__dynamic_cast + 136
>
> 5. Is there anything that I missed? I'm not sure where I went wrong. Any
> suggestions would be appreciated.
>

Mime
View raw message