arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From swizz one <soswi...@gmail.com>
Subject Re: Sending arrow via socket
Date Sat, 02 May 2020 05:07:37 GMT
Thank you very much, Your answer has been really helpful. Can I ask a
question?
Is it necessary to create a buffer to send data via socket, is, for example
I create the schemas, do I have to create a buffer to store the data and
then send he that data?,
Also is this method more effective compared to writing data to a arrow file
and reading from that arrow file in the javascript program?
Finally, what are the option that are available for passing data from one
program language to another using arrow?

Sorry, for the questions, I am a bit new to programming.

El vie., 1 may. 2020 a las 23:30, Sutou Kouhei (<kou@clear-code.com>)
escribió:

> Hi,
>
> Do you want to use Apache Arrow C GLib instead of Apache
> Arrow C++, right?
>
> We don't provide a memory pool in Apache Arrow C GLib API.
>
> You can create a resizable buffer and output serialize data
> to it:
>
> ---
> GError *error = NULL;
> GArrowResizableBuffer *buffer = garrow_resizable_buffer_new(1024, &error);
> if (!buffer) {
>   g_print("error: %s\n", error->message);
>   g_error_free(error);
>   return;
> }
>
> GArrowBufferOutputStream *output = garrow_buffer_output_stream_new(buffer);
>
> GArrowRecordBatchStreamWriter *writer =
>   garrow_record_batch_stream_writer_new(GARROW_OUTPUT_STREAM(output),
>                                         schema, /* You need to create this
> */
>                                         &error);
> if (!writer) {
>   g_print("error: %s\n", error->message);
>   g_error_free(error);
>   g_object_unref(output);
>   g_object_unref(buffer);
>   return;
> }
>
> if (!garrow_record_batch_writer_write_record_batch(
>        GARROW_RECORD_BATCH_WRITER(writer),
>        record_batch, /* You need to create this */
>        &error)) {
>   g_print("error: %s\n", error->message);
>   g_error_free(error);
>   g_object_unref(writer);
>   g_object_unref(output);
>   g_object_unref(buffer);
>   return;
> }
>
> if (!garrow_record_batch_writer_close(
>        GARROW_RECORD_BATCH_WRITER(writer),
>        &error)) {
>   g_print("error: %s\n", error->message);
>   g_error_free(error);
>   g_object_unref(writer);
>   g_object_unref(output);
>   g_object_unref(buffer);
>   return;
> }
>
> GBytes *data = garrow_buffer_get_data(GARROW_BUFFER(buffer));
> gsize data_size;
> gconstpointer data_raw = g_bytes_get_data(data, &data_size);
> write(websocket_fd, data_raw, data_size);
> g_bytes_unref(data);
>
> g_object_unref(writer);
> g_object_unref(output);
> g_object_unref(buffer);
> ---
>
>
> Thanks,
> --
> kou
>
> In <CACKQ_V3_yeuNn5TzPiF1RS=6WmwjL5jR4Vgx39GpQ3FvMgEqzg@mail.gmail.com>
>   "Sending arrow via socket" on Fri, 1 May 2020 18:12:34 +0200,
>   swizz one <soswizzy@gmail.com> wrote:
>
> > Please, I am currently working on a project that require sending data
> from
> > an c program to perpespective(javascript) via socket from c. Since
> > perpespective works with arrow, it was a perfect choice. Is it possible
> to
> > send table at x interval from c to javascript via websocket without
> > creating a an arrow binary format file?
> > How to you create a memory pool primotive with c, without having to size
> to
> > have a fixed sized on the array like the memory pool in c++.
> >
> > Your's Faithfully,
> > Thank you
>

Mime
View raw message