arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: Streaming Arrow files in Python to JS
Date Thu, 31 Oct 2019 17:50:11 GMT
hi,

We have a well-defined stream format which can be sent over any
network transport

http://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format

Once you decide on your network transport, you will want to use the
stream writer class from Python to send the byte stream to a reader in
JavaScript. I don't know any examples to point you to to get started
though.

- Wes

On Thu, Oct 31, 2019 at 3:19 AM Akshay Verma <akshay.verma@tuta.io> wrote:
>
>
> Hello,
>
> I am very excited about the Arrow project and want to introduce it in the new application
that we are developing. The first place where we can easily introduce is the communication
between backend process and frontend visualisation. Currently we are doing it as JSON. We
can currently convert that JSON to an arrow Table/RecordBatch and save it locally in Python.
>
> Ideally, we would like to stream the big JSON data from Python to the JS application
and update the visualisation as the streamed data comes in.
>
> Is there any example, or an intended approach we could take to implement this in our
project?
>
> Thanks for the project.
>
> Kind Regards,
> Akshay Verma

Mime
View raw message