arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Cutler <cutl...@gmail.com>
Subject Re: [Java] Flight fails to handle list vector
Date Thu, 14 Jan 2021 19:22:42 GMT
Hi John, could you include the error with stacktrace?

On Sat, Jan 9, 2021 at 9:34 PM John Peterson <donttrytocontactme2@gmail.com>
wrote:

> I believe I'm running into a bug with Flight but I'd like to confirm and
> get some advice on a potential fix. I'm not sure where to look or what
> could be causing it.
>
> The code in question simply uploads a one-element List<Integer> to the
> example server, fetches it from the server, and attempts to append the data
> from the server to a new VectorSchemaRoot. It fails in the same way
> regardless of whether or not I construct a VectorSchemaRoot instance.
>
> Likewise, the data from the server can't be written out with the JSON
> writer, it'll fail in the same way. However, changing the data from a
> ListVector to an IntVector causes it to succeed.
>
> Any help would be appreciated.
>
> Thanks,
> John
>
> Code in question:
> // Set up the server and client
> BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
> Location l = Location.forGrpcInsecure(FlightTestUtil.LOCALHOST, 12233);
> ExampleFlightServer server = new ExampleFlightServer(allocator, l);
> server.start();
> FlightClient client = FlightClient.builder(allocator, l).build();
>
> // Write a one-element List<Integer>
> ListVector listVector = ListVector.empty("list", allocator);
> UnionListWriter writer = listVector.getWriter();
> writer.startList();
> writer.integer().writeInt(1);
> writer.endList();
> writer.setValueCount(1);
>
> // Send that data to the server
> VectorSchemaRoot root = VectorSchemaRoot.of(listVector);
> ClientStreamListener listener =
> client.startPut(FlightDescriptor.path("test"), root, new
> AsyncPutListener());
> root.setRowCount(1);
> listener.putNext();
> root.clear();
> listener.completed();
>
> // wait for ack to avoid memory leaks.
> listener.getResult();
>
> // Attempt to read it back
> FlightInfo info = client.getInfo(FlightDescriptor.path("test"));
> try (final FlightStream stream =
> client.getStream(info.getEndpoints().get(0).getTicket())) {
>   VectorSchemaRoot newRoot = stream.getRoot();
>   while (stream.next()) {
>     // Copying into an entirely new VectorSchemaRoot fails
>     try {
>       ListVector newList = ListVector.empty("list", allocator);
>
> newList.addOrGetVector(FieldType.nullable(Types.MinorType.INT.getType()));
>       VectorSchemaRoot copyRoot = VectorSchemaRoot.of(newList);
>       VectorSchemaRootAppender.append(copyRoot, newRoot);
>     } catch (IndexOutOfBoundsException e) {
>       System.err.println("Expected IOOBE caught");
>     }
>
>     // The same is true if we try to copy the data from the server to our
> VectorSchemaRoot
>     try {
>       VectorSchemaRootAppender.append(root, newRoot);
>     } catch (IndexOutOfBoundsException e) {
>       System.err.println("Expected IOOBE caught again");
>       throw e;
>     }
>
>     root.clear();
>     newRoot.clear();
>   }
> }
>

Mime
View raw message