arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sutou Kouhei <...@clear-code.com>
Subject Re: Sending arrow via socket
Date Fri, 01 May 2020 21:29:54 GMT
Hi,

Do you want to use Apache Arrow C GLib instead of Apache
Arrow C++, right?

We don't provide a memory pool in Apache Arrow C GLib API.

You can create a resizable buffer and output serialize data
to it:

---
GError *error = NULL;
GArrowResizableBuffer *buffer = garrow_resizable_buffer_new(1024, &error);
if (!buffer) {
  g_print("error: %s\n", error->message);
  g_error_free(error);
  return;
}

GArrowBufferOutputStream *output = garrow_buffer_output_stream_new(buffer);

GArrowRecordBatchStreamWriter *writer =
  garrow_record_batch_stream_writer_new(GARROW_OUTPUT_STREAM(output),
                                        schema, /* You need to create this */
                                        &error);
if (!writer) {
  g_print("error: %s\n", error->message);
  g_error_free(error);
  g_object_unref(output);
  g_object_unref(buffer);
  return;
}

if (!garrow_record_batch_writer_write_record_batch(
       GARROW_RECORD_BATCH_WRITER(writer),
       record_batch, /* You need to create this */
       &error)) {
  g_print("error: %s\n", error->message);
  g_error_free(error);
  g_object_unref(writer);
  g_object_unref(output);
  g_object_unref(buffer);
  return;
}

if (!garrow_record_batch_writer_close(
       GARROW_RECORD_BATCH_WRITER(writer),
       &error)) {
  g_print("error: %s\n", error->message);
  g_error_free(error);
  g_object_unref(writer);
  g_object_unref(output);
  g_object_unref(buffer);
  return;
}

GBytes *data = garrow_buffer_get_data(GARROW_BUFFER(buffer));
gsize data_size;
gconstpointer data_raw = g_bytes_get_data(data, &data_size);
write(websocket_fd, data_raw, data_size);
g_bytes_unref(data);

g_object_unref(writer);
g_object_unref(output);
g_object_unref(buffer);
---


Thanks,
--
kou

In <CACKQ_V3_yeuNn5TzPiF1RS=6WmwjL5jR4Vgx39GpQ3FvMgEqzg@mail.gmail.com>
  "Sending arrow via socket" on Fri, 1 May 2020 18:12:34 +0200,
  swizz one <soswizzy@gmail.com> wrote:

> Please, I am currently working on a project that require sending data from
> an c program to perpespective(javascript) via socket from c. Since
> perpespective works with arrow, it was a perfect choice. Is it possible to
> send table at x interval from c to javascript via websocket without
> creating a an arrow binary format file?
> How to you create a memory pool primotive with c, without having to size to
> have a fixed sized on the array like the memory pool in c++.
> 
> Your's Faithfully,
> Thank you

Mime
View raw message