arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saloni Udani <saloniudani.t...@gmail.com>
Subject Arrow java implementation: Compatible IO streams.
Date Wed, 04 Nov 2020 08:43:09 GMT
Hello,
I have a use case where I want to write an arrow batch to my existing
output stream (custom stream extending java.io.OutputStream)  and reading
from my existing input stream (custom stream extending
java.io.InputStream).  I used ArrowStreamWriter and ArrowStreamReader but
on the reader side I get NPE while getting VectorSchemaRoot.

Caused by: java.lang.NullPointerException: null
at
org.apache.arrow.vector.types.pojo.Field.mutateOriginalNameIfNeeded(Field.java:188)
at org.apache.arrow.vector.types.pojo.Field.convertField(Field.java:172)
at org.apache.arrow.vector.types.pojo.Schema.convertSchema(Schema.java:87)
at
org.apache.arrow.vector.ipc.message.MessageSerializer.deserializeSchema(MessageSerializer.java:201)
at
org.apache.arrow.vector.ipc.ArrowStreamReader.readSchema(ArrowStreamReader.java:174)
at org.apache.arrow.vector.ipc.ArrowReader.initialize(ArrowReader.java:170)
at
org.apache.arrow.vector.ipc.ArrowReader.ensureInitialized(ArrowReader.java:161)
at
org.apache.arrow.vector.ipc.ArrowReader.getVectorSchemaRoot(ArrowReader.java:63)




Kindly guide what am I doing wrong? I believe this to be an issue of IO
streams. Is there any prerequisite to which IO streams can be used with
Arrow reader and writer?

Also how can I validate if the output generated by Arrow writer is in valid
format?


Regards
Saloni Udani

Mime
View raw message