arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Micah Kornfield <emkornfi...@gmail.com>
Subject Re: Arrow java implementation: Compatible IO streams.
Date Fri, 13 Nov 2020 06:42:25 GMT
Hi Saloni,
Sample code for how you are using the Arrow APIs are useful.

Also how can I validate if the output generated by Arrow writer is in valid
> format?

Being able to read it back would be the best mechanism here?  Are you think
of unit tests or in production?  Or something else?

Thanks,
Micah

On Wed, Nov 4, 2020 at 12:43 AM Saloni Udani <saloniudani.tech@gmail.com>
wrote:

> Hello,
> I have a use case where I want to write an arrow batch to my existing
> output stream (custom stream extending java.io.OutputStream)  and reading
> from my existing input stream (custom stream extending
> java.io.InputStream).  I used ArrowStreamWriter and ArrowStreamReader but
> on the reader side I get NPE while getting VectorSchemaRoot.
>
> Caused by: java.lang.NullPointerException: null
> at
> org.apache.arrow.vector.types.pojo.Field.mutateOriginalNameIfNeeded(Field.java:188)
> at org.apache.arrow.vector.types.pojo.Field.convertField(Field.java:172)
> at org.apache.arrow.vector.types.pojo.Schema.convertSchema(Schema.java:87)
> at
> org.apache.arrow.vector.ipc.message.MessageSerializer.deserializeSchema(MessageSerializer.java:201)
> at
> org.apache.arrow.vector.ipc.ArrowStreamReader.readSchema(ArrowStreamReader.java:174)
> at org.apache.arrow.vector.ipc.ArrowReader.initialize(ArrowReader.java:170)
> at
> org.apache.arrow.vector.ipc.ArrowReader.ensureInitialized(ArrowReader.java:161)
> at
> org.apache.arrow.vector.ipc.ArrowReader.getVectorSchemaRoot(ArrowReader.java:63)
>
>
>
>
> Kindly guide what am I doing wrong? I believe this to be an issue of IO
> streams. Is there any prerequisite to which IO streams can be used with
> Arrow reader and writer?
>
> Also how can I validate if the output generated by Arrow writer is in
> valid format?
>
>
> Regards
> Saloni Udani
>

Mime
View raw message