arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Le Dem (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ARROW-414) [Java] "Buffer too large to resize to ..." error
Date Fri, 09 Dec 2016 19:31:59 GMT

    [ https://issues.apache.org/jira/browse/ARROW-414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736089#comment-15736089
] 

Julien Le Dem commented on ARROW-414:
-------------------------------------

So it's complaining that the padding is more than 64 bytes.
based on fieldNode.getLength() the buffer should be of length 44 ((10 values + 1) x 4 bytes)
I added a check that padding should be no more than 64 bytes so 128 is too big.
(and I guess I should verify that it is on 64 bytes boundaries instead of < 64)

Is the padding rule correct?
How strict do we want to be?
In general I prefer to be stricter to catch bugs.



> [Java] "Buffer too large to resize to ..." error
> ------------------------------------------------
>
>                 Key: ARROW-414
>                 URL: https://issues.apache.org/jira/browse/ARROW-414
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Java - Vectors
>            Reporter: Wes McKinney
>         Attachments: struct_example.arrow, struct_example.json
>
>
> On the latest branch for ARROW-394 (after rebasing on all the fixes so far), I have the
error
> {code}
> Incompatible files
> Could not load buffers for field f2: Utf8. error message: Buffer too large to resize
to 44: 128
> 13:41:19.356 [main] ERROR org.apache.arrow.tools.Integration - Incompatible files
> java.lang.IllegalArgumentException: Could not load buffers for field f2: Utf8. error
message: Buffer too large to resize to 44: 128
> 	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:84) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> 	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:94) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> 	at org.apache.arrow.vector.VectorLoader.load(VectorLoader.java:63) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> 	at org.apache.arrow.tools.Integration$Command$3.execute(Integration.java:156) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> 	at org.apache.arrow.tools.Integration.run(Integration.java:212) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> 	at org.apache.arrow.tools.Integration.main(Integration.java:61) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> Caused by: java.lang.IllegalArgumentException: Buffer too large to resize to 44: 128
> 	at org.apache.arrow.vector.BaseDataValueVector.truncateBufferBasedOnSize(BaseDataValueVector.java:55)
~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> 	at org.apache.arrow.vector.NullableVarCharVector.loadFieldBuffers(NullableVarCharVector.java:142)
~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> 	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:82) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
> 	... 5 common frames omitted
> {code}
> See attached JSON and C++-generated Arrow file



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message