carbondata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kunal Kapoor <kunalkapoor...@gmail.com>
Subject Re: java.lang.NegativeArraySizeException occurred when compact
Date Wed, 17 Oct 2018 09:32:06 GMT
Hi,
I have a few question regarding this Exception:
1. Does the table have a string columns for which the length of the data is
exceeding 32k characters?
2. Are you able to query(select *) on the table successfully?
3. Can you share the schema of the table?

Meanwhile i am looking into the possibilities of any other thread clearing
the MemoryBlock.

Regards
Kunal Kapoor

On Tue, Oct 16, 2018 at 1:31 PM xm_zzc <441586683@qq.com> wrote:

> Hi Babu:
>   Thanks for your reply.
>   I set enable.unsafe.in.query.processing=false  and
> enable.unsafe.columnpage=false , and test failed still.
>   I think the issue I met is not related to MemoryBlock which is cleaned by
> some other thread. As the test steps I mentioned above, I copy the wrong
> segment and use SDKReader to read data, it failed too, the error  message
> is
> following:
> *java.lang.RuntimeException: java.lang.IllegalArgumentException
>         at
>
> org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk.convertToDimColDataChunkWithOutCache(DimensionRawColumnChunk.java:120)
>         at
>
> org.apache.carbondata.core.scan.result.BlockletScannedResult.fillDataChunks(BlockletScannedResult.java:355)
>         at
>
> org.apache.carbondata.core.scan.result.BlockletScannedResult.hasNext(BlockletScannedResult.java:559)
>         at
>
> org.apache.carbondata.core.scan.collector.impl.DictionaryBasedResultCollector.collectResultInRow(DictionaryBasedResultCollector.java:137)
>         at
>
> org.apache.carbondata.core.scan.processor.DataBlockIterator.next(DataBlockIterator.java:109)
>         at
>
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.getBatchResult(DetailQueryResultIterator.java:49)
>         at
>
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:41)
>         at
>
> org.apache.carbondata.core.scan.result.iterator.DetailQueryResultIterator.next(DetailQueryResultIterator.java:1)
>         at
>
> org.apache.carbondata.core.scan.result.iterator.ChunkRowIterator.hasNext(ChunkRowIterator.java:58)
>         at
>
> org.apache.carbondata.hadoop.CarbonRecordReader.nextKeyValue(CarbonRecordReader.java:104)
>         at
> org.apache.carbondata.sdk.file.CarbonReader.hasNext(CarbonReader.java:71)
>         at
> cn.xm.zzc.carbonsdktest.CarbonSDKTest.main(CarbonSDKTest.java:68)
> Caused by: java.lang.IllegalArgumentException
>         at java.nio.Buffer.position(Buffer.java:244)
>         at
>
> org.apache.carbondata.core.datastore.chunk.store.impl.unsafe.UnsafeVariableLengthDimensionDataChunkStore.putArray(UnsafeVariableLengthDimensionDataChunkStore.java:97)
>         at
>
> org.apache.carbondata.core.datastore.chunk.impl.VariableLengthDimensionColumnPage.<init>(VariableLengthDimensionColumnPage.java:58)
>         at
>
> org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.CompressedDimensionChunkFileBasedReaderV3.decodeDimensionLegacy(CompressedDimensionChunkFileBasedReaderV3.java:325)
>         at
>
> org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.CompressedDimensionChunkFileBasedReaderV3.decodeDimension(CompressedDimensionChunkFileBasedReaderV3.java:266)
>         at
>
> org.apache.carbondata.core.datastore.chunk.reader.dimension.v3.CompressedDimensionChunkFileBasedReaderV3.decodeColumnPage(CompressedDimensionChunkFileBasedReaderV3.java:224)
>         at
>
> org.apache.carbondata.core.datastore.chunk.impl.DimensionRawColumnChunk.convertToDimColDataChunkWithOutCache(DimensionRawColumnChunk.java:118)
>         ... 11 more*
>
> when error occurred, the values of some parameters in
> UnsafeVariableLengthDimensionDataChunkStore.putArray are as following :
>
> buffer.limit=192000
> buffer.cap=192000
> startOffset=300289
> numberOfRows=32000
> this.dataPointersOffsets=288000
>
> startOffset is bigger than buffer.limit, so error occurred.
>
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message