asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hussain Towaileb (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (ASTERIXDB-2717) Querying external dataset fails when further data read is needed before decoding
Date Tue, 21 Apr 2020 12:41:00 GMT

     [ https://issues.apache.org/jira/browse/ASTERIXDB-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hussain Towaileb resolved ASTERIXDB-2717.
-----------------------------------------
    Resolution: Fixed

Fix reference:

[https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5724]

> Querying external dataset fails when further data read is needed before decoding
> --------------------------------------------------------------------------------
>
>                 Key: ASTERIXDB-2717
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-2717
>             Project: Apache AsterixDB
>          Issue Type: Bug
>          Components: EXT - External data
>    Affects Versions: 0.9.5
>            Reporter: Hussain Towaileb
>            Assignee: Hussain Towaileb
>            Priority: Major
>             Fix For: 0.9.5
>
>
> In some scenarios, some characters are represented by multiple bytes, and the bytes are
partially read (rest of the bytes is in the next read). In this case, the decoder will fail
to decode, and will wait for the next read to have all the bytes together then try to decode
it.
>  
> In the above scenario, the ByteBuffer limit is not being reset before reading again,
which could lead to the position being greater than the limit and throwing an exception.
>  
> For example:
>  
> {code:java}
> org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.IllegalArgumentException:
newPosition > limit: (4096 > 1048)
>  at org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:51)
~[hyracks-api.jar:6.6.0-11077]
>  at org.apache.asterix.external.dataflow.RecordDataFlowController.start(RecordDataFlowController.java:59)
~[asterix-external-data.jar:6.6.0-11077]
>  at org.apache.asterix.external.dataset.adapter.GenericAdapter.start(GenericAdapter.java:36)
~[asterix-external-data.jar:6.6.0-11077]
>  at org.apache.asterix.external.operators.ExternalScanOperatorDescriptor$1.initialize(ExternalScanOperatorDescriptor.java:62)
~[asterix-external-data.jar:6.6.0-11077]
>  at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$8(SuperActivityOperatorNodePushable.java:228)
~[hyracks-api.jar:6.6.0-11077]
>  at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
>  at java.lang.Thread.run(Unknown Source) [?:?]
>  Caused by: java.lang.IllegalArgumentException: newPosition > limit: (4096 > 1048)
>  at java.nio.Buffer.createPositionException(Unknown Source) ~[?:?]
>  at java.nio.Buffer.position(Unknown Source) ~[?:?]
>  at java.nio.ByteBuffer.position(Unknown Source) ~[?:?]
>  at java.nio.ByteBuffer.position(Unknown Source) ~[?:?]
>  at org.apache.asterix.external.input.stream.AsterixInputStreamReader.read(AsterixInputStreamReader.java:103)
~[asterix-external-data.jar:6.6.0-11077]
>  at org.apache.asterix.external.input.stream.AsterixInputStreamReader.read(AsterixInputStreamReader.java:67)
~[asterix-external-data.jar:6.6.0-11077]
>  at org.apache.asterix.external.input.record.reader.stream.LineRecordReader.hasNext(LineRecordReader.java:101)
~[asterix-external-data.jar:6.6.0-11077]
>  at org.apache.asterix.external.dataflow.RecordDataFlowController.start(RecordDataFlowController.java:49)
~[asterix-external-data.jar:6.6.0-11077]
>  ... 7 more{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message