arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Peterson <donttrytocontact...@gmail.com>
Subject Re: [Java] Flight fails to handle list vector
Date Mon, 15 Feb 2021 02:20:34 GMT
Hi Bryan,

This is the stacktrace I get:

java.lang.IndexOutOfBoundsException: index: 0, length: 4 (expected:
range(0, 0))
at org.apache.arrow.memory.ArrowBuf.checkIndexD(ArrowBuf.java:318)
at org.apache.arrow.memory.ArrowBuf.chk(ArrowBuf.java:305)
at org.apache.arrow.memory.ArrowBuf.getInt(ArrowBuf.java:424)
at org.apache.arrow.vector.util.VectorAppender.visit(VectorAppender.java:97)
at org.apache.arrow.vector.util.VectorAppender.visit(VectorAppender.java:45)
at
org.apache.arrow.vector.BaseVariableWidthVector.accept(BaseVariableWidthVector.java:1402)
at
org.apache.arrow.vector.util.VectorAppender.visit(VectorAppender.java:233)
at org.apache.arrow.vector.util.VectorAppender.visit(VectorAppender.java:45)
at org.apache.arrow.vector.complex.ListVector.accept(ListVector.java:449)
at
org.apache.arrow.vector.util.VectorSchemaRootAppender.append(VectorSchemaRootAppender.java:67)
at
org.apache.arrow.vector.util.VectorSchemaRootAppender.append(VectorSchemaRootAppender.java:81)

Thanks for your help.

On Thu, Jan 14, 2021 at 2:23 PM Bryan Cutler <cutlerb@gmail.com> wrote:

> Hi John, could you include the error with stacktrace?
>
> On Sat, Jan 9, 2021 at 9:34 PM John Peterson <
> donttrytocontactme2@gmail.com> wrote:
>
>> I believe I'm running into a bug with Flight but I'd like to confirm and
>> get some advice on a potential fix. I'm not sure where to look or what
>> could be causing it.
>>
>> The code in question simply uploads a one-element List<Integer> to the
>> example server, fetches it from the server, and attempts to append the data
>> from the server to a new VectorSchemaRoot. It fails in the same way
>> regardless of whether or not I construct a VectorSchemaRoot instance.
>>
>> Likewise, the data from the server can't be written out with the JSON
>> writer, it'll fail in the same way. However, changing the data from a
>> ListVector to an IntVector causes it to succeed.
>>
>> Any help would be appreciated.
>>
>> Thanks,
>> John
>>
>> Code in question:
>> // Set up the server and client
>> BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
>> Location l = Location.forGrpcInsecure(FlightTestUtil.LOCALHOST, 12233);
>> ExampleFlightServer server = new ExampleFlightServer(allocator, l);
>> server.start();
>> FlightClient client = FlightClient.builder(allocator, l).build();
>>
>> // Write a one-element List<Integer>
>> ListVector listVector = ListVector.empty("list", allocator);
>> UnionListWriter writer = listVector.getWriter();
>> writer.startList();
>> writer.integer().writeInt(1);
>> writer.endList();
>> writer.setValueCount(1);
>>
>> // Send that data to the server
>> VectorSchemaRoot root = VectorSchemaRoot.of(listVector);
>> ClientStreamListener listener =
>> client.startPut(FlightDescriptor.path("test"), root, new
>> AsyncPutListener());
>> root.setRowCount(1);
>> listener.putNext();
>> root.clear();
>> listener.completed();
>>
>> // wait for ack to avoid memory leaks.
>> listener.getResult();
>>
>> // Attempt to read it back
>> FlightInfo info = client.getInfo(FlightDescriptor.path("test"));
>> try (final FlightStream stream =
>> client.getStream(info.getEndpoints().get(0).getTicket())) {
>>   VectorSchemaRoot newRoot = stream.getRoot();
>>   while (stream.next()) {
>>     // Copying into an entirely new VectorSchemaRoot fails
>>     try {
>>       ListVector newList = ListVector.empty("list", allocator);
>>
>> newList.addOrGetVector(FieldType.nullable(Types.MinorType.INT.getType()));
>>       VectorSchemaRoot copyRoot = VectorSchemaRoot.of(newList);
>>       VectorSchemaRootAppender.append(copyRoot, newRoot);
>>     } catch (IndexOutOfBoundsException e) {
>>       System.err.println("Expected IOOBE caught");
>>     }
>>
>>     // The same is true if we try to copy the data from the server to our
>> VectorSchemaRoot
>>     try {
>>       VectorSchemaRootAppender.append(root, newRoot);
>>     } catch (IndexOutOfBoundsException e) {
>>       System.err.println("Expected IOOBE caught again");
>>       throw e;
>>     }
>>
>>     root.clear();
>>     newRoot.clear();
>>   }
>> }
>>
>

Mime
View raw message