arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From swizz one <soswi...@gmail.com>
Subject Re: Sending arrow via socket
Date Mon, 04 May 2020 12:18:58 GMT
How do you read that buffer data from another process with a different
programming language? I am referring to the gconstpointer data type?
Thank you very much

On Fri, May 1, 2020, 23:30 Sutou Kouhei <kou@clear-code.com> wrote:

> Hi,
>
> Do you want to use Apache Arrow C GLib instead of Apache
> Arrow C++, right?
>
> We don't provide a memory pool in Apache Arrow C GLib API.
>
> You can create a resizable buffer and output serialize data
> to it:
>
> ---
> GError *error = NULL;
> GArrowResizableBuffer *buffer = garrow_resizable_buffer_new(1024, &error);
> if (!buffer) {
>   g_print("error: %s\n", error->message);
>   g_error_free(error);
>   return;
> }
>
> GArrowBufferOutputStream *output = garrow_buffer_output_stream_new(buffer);
>
> GArrowRecordBatchStreamWriter *writer =
>   garrow_record_batch_stream_writer_new(GARROW_OUTPUT_STREAM(output),
>                                         schema, /* You need to create this
> */
>                                         &error);
> if (!writer) {
>   g_print("error: %s\n", error->message);
>   g_error_free(error);
>   g_object_unref(output);
>   g_object_unref(buffer);
>   return;
> }
>
> if (!garrow_record_batch_writer_write_record_batch(
>        GARROW_RECORD_BATCH_WRITER(writer),
>        record_batch, /* You need to create this */
>        &error)) {
>   g_print("error: %s\n", error->message);
>   g_error_free(error);
>   g_object_unref(writer);
>   g_object_unref(output);
>   g_object_unref(buffer);
>   return;
> }
>
> if (!garrow_record_batch_writer_close(
>        GARROW_RECORD_BATCH_WRITER(writer),
>        &error)) {
>   g_print("error: %s\n", error->message);
>   g_error_free(error);
>   g_object_unref(writer);
>   g_object_unref(output);
>   g_object_unref(buffer);
>   return;
> }
>
> GBytes *data = garrow_buffer_get_data(GARROW_BUFFER(buffer));
> gsize data_size;
> gconstpointer data_raw = g_bytes_get_data(data, &data_size);
> write(websocket_fd, data_raw, data_size);
> g_bytes_unref(data);
>
> g_object_unref(writer);
> g_object_unref(output);
> g_object_unref(buffer);
> ---
>
>
> Thanks,
> --
> kou
>
> In <CACKQ_V3_yeuNn5TzPiF1RS=6WmwjL5jR4Vgx39GpQ3FvMgEqzg@mail.gmail.com>
>   "Sending arrow via socket" on Fri, 1 May 2020 18:12:34 +0200,
>   swizz one <soswizzy@gmail.com> wrote:
>
> > Please, I am currently working on a project that require sending data
> from
> > an c program to perpespective(javascript) via socket from c. Since
> > perpespective works with arrow, it was a perfect choice. Is it possible
> to
> > send table at x interval from c to javascript via websocket without
> > creating a an arrow binary format file?
> > How to you create a memory pool primotive with c, without having to size
> to
> > have a fixed sized on the array like the memory pool in c++.
> >
> > Your's Faithfully,
> > Thank you
>

Mime
View raw message