arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Calder, Matthew" <mcal...@xbktrading.com>
Subject Copying and memory ownership question
Date Wed, 08 Jan 2020 12:49:29 GMT
Hi,

I created a minimal cython interface to c++ and I am unsure of whether or not memory is copied
and how it is eventually freed. My files are:

--- xbk.hpp ---
#pragma once
#include <arrow/api.h>
namespace xbk {
    std::shared_ptr<arrow::Array> makeArray();
}

--- xbk.cpp ---
#include <vector>
#include "xbk.hpp"
namespace xbk {
    std::shared_ptr<arrow::Array> makeArray()
    {
        std::vector<std::string> v = {"A", "B", "C"};
        arrow::StringBuilder builder;
        builder.AppendValues(v);
        std::shared_ptr<arrow::Array> array;
        builder.Finish(&array);
        return array;
    }
}

--- xbk.pxd ---
from pyarrow.lib cimport *
cdef extern from "xbk.cpp":
    pass
cdef extern from "xbk.hpp" namespace "xbk":
    cdef shared_ptr[CArray] makeArray()

--- xbk_arrow.pyx ---
# distutils: language = c++
from xbk cimport makeArray
from pyarrow.lib cimport *

def makeArrayWrapper():
    a = makeArray()
    return pyarrow_wrap_array(a)

--- caller.py ---
from xbk_arrow import makeArrayWrapper
a = makeArrayWrapper()
f"{a[0]} {a[1]} {a[2]}"


My questions are: when calling makeArrayWrapper from caller.py is the Array created within
the makeArray function copied when it is converted into a python object? Also, is the memory
freed by the python gc and/or the c++ lib in a timely way? If there is copying or leaking
in the above setup,  what is the correct way to pass arrow objects created in c++ libraries
back to python without copying or leaking? I read over https://arrow.apache.org/docs/python/extending.html
but I am still unsure. Thanks for any help,

Matt


The information contained in this e-mail may be confidential and is intended solely for the
use of the named addressee.

Access, copying or re-use of the e-mail or any information contained therein by any other
person is not authorized.

If you are not the intended recipient please notify us immediately by returning the e-mail
to the originator.

Disclaimer Version MB.US.1
Mime
View raw message