arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From swizz one <soswi...@gmail.com>
Subject Re: Sending arrow via socket
Date Sat, 02 May 2020 05:46:55 GMT
And also, instead of sending the pointer, can you send the buffer directly?

El sáb., 2 may. 2020 a las 7:07, swizz one (<soswizzy@gmail.com>) escribió:

> Thank you very much, Your answer has been really helpful. Can I ask a
> question?
> Is it necessary to create a buffer to send data via socket, is, for
> example I create the schemas, do I have to create a buffer to store the
> data and then send he that data?,
> Also is this method more effective compared to writing data to a arrow
> file and reading from that arrow file in the javascript program?
> Finally, what are the option that are available for passing data from one
> program language to another using arrow?
>
> Sorry, for the questions, I am a bit new to programming.
>
> El vie., 1 may. 2020 a las 23:30, Sutou Kouhei (<kou@clear-code.com>)
> escribió:
>
>> Hi,
>>
>> Do you want to use Apache Arrow C GLib instead of Apache
>> Arrow C++, right?
>>
>> We don't provide a memory pool in Apache Arrow C GLib API.
>>
>> You can create a resizable buffer and output serialize data
>> to it:
>>
>> ---
>> GError *error = NULL;
>> GArrowResizableBuffer *buffer = garrow_resizable_buffer_new(1024, &error);
>> if (!buffer) {
>>   g_print("error: %s\n", error->message);
>>   g_error_free(error);
>>   return;
>> }
>>
>> GArrowBufferOutputStream *output =
>> garrow_buffer_output_stream_new(buffer);
>>
>> GArrowRecordBatchStreamWriter *writer =
>>   garrow_record_batch_stream_writer_new(GARROW_OUTPUT_STREAM(output),
>>                                         schema, /* You need to create
>> this */
>>                                         &error);
>> if (!writer) {
>>   g_print("error: %s\n", error->message);
>>   g_error_free(error);
>>   g_object_unref(output);
>>   g_object_unref(buffer);
>>   return;
>> }
>>
>> if (!garrow_record_batch_writer_write_record_batch(
>>        GARROW_RECORD_BATCH_WRITER(writer),
>>        record_batch, /* You need to create this */
>>        &error)) {
>>   g_print("error: %s\n", error->message);
>>   g_error_free(error);
>>   g_object_unref(writer);
>>   g_object_unref(output);
>>   g_object_unref(buffer);
>>   return;
>> }
>>
>> if (!garrow_record_batch_writer_close(
>>        GARROW_RECORD_BATCH_WRITER(writer),
>>        &error)) {
>>   g_print("error: %s\n", error->message);
>>   g_error_free(error);
>>   g_object_unref(writer);
>>   g_object_unref(output);
>>   g_object_unref(buffer);
>>   return;
>> }
>>
>> GBytes *data = garrow_buffer_get_data(GARROW_BUFFER(buffer));
>> gsize data_size;
>> gconstpointer data_raw = g_bytes_get_data(data, &data_size);
>> write(websocket_fd, data_raw, data_size);
>> g_bytes_unref(data);
>>
>> g_object_unref(writer);
>> g_object_unref(output);
>> g_object_unref(buffer);
>> ---
>>
>>
>> Thanks,
>> --
>> kou
>>
>> In <CACKQ_V3_yeuNn5TzPiF1RS=6WmwjL5jR4Vgx39GpQ3FvMgEqzg@mail.gmail.com>
>>   "Sending arrow via socket" on Fri, 1 May 2020 18:12:34 +0200,
>>   swizz one <soswizzy@gmail.com> wrote:
>>
>> > Please, I am currently working on a project that require sending data
>> from
>> > an c program to perpespective(javascript) via socket from c. Since
>> > perpespective works with arrow, it was a perfect choice. Is it possible
>> to
>> > send table at x interval from c to javascript via websocket without
>> > creating a an arrow binary format file?
>> > How to you create a memory pool primotive with c, without having to
>> size to
>> > have a fixed sized on the array like the memory pool in c++.
>> >
>> > Your's Faithfully,
>> > Thank you
>>
>

Mime
View raw message