carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacky Li (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CARBONDATA-1267) Failure in data loading due to bugs in delta-integer-codec
Date Mon, 10 Jul 2017 13:39:00 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jacky Li resolved CARBONDATA-1267.
----------------------------------
       Resolution: Fixed
    Fix Version/s: 1.2.0

> Failure in data loading due to bugs in delta-integer-codec
> ----------------------------------------------------------
>
>                 Key: CARBONDATA-1267
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1267
>             Project: CarbonData
>          Issue Type: Bug
>          Components: core, data-load
>    Affects Versions: 1.1.0
>         Environment: suse11.3
>            Reporter: xuchuanyin
>            Priority: Critical
>             Fix For: 1.2.0
>
>          Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> ## BRIEF
> when encoding column value by DeltaIntegerCodec, SHORT_INT may be a target type, but
this value doesnot exist in swith-case branch, so exception will be thrown in some case.
> ## ANALYZE´╝Ü
> This error occurs in particular situation. In my case, there is a measure column called`
BEGIN_TIME` and its datatype is `LONG`. When encoding this measure, carbondata use the statistic
information `(min: 1497376581, max: 1497423838, decimal: 0 )` and finally decide to use `DeltaIntegerCodec`
encoder with source datatype `LONG` and target datatype `SHORT_INT(a internal datatype using
three bytes)`. But the encode method in `DeltaIntegerCodec` does not consider this situation,
so exception will be thrown.
> # ERROR
> Detail error message is shown below:
> java.lang.RuntimeException: internal error: org.apache.carbondata.core.datastore.page.encoding.DeltaIntegerCodec[src
type: LONG, target type: SHORT_INT, stats(min: 1497376581, max: 1497423838, decimal: 0 )]
> 	at org.apache.carbondata.core.datastore.page.encoding.DeltaIntegerCodec$1.encode(DeltaIntegerCodec.java:147)
> 	at org.apache.carbondata.core.datastore.page.SafeFixLengthColumnPage.encode(SafeFixLengthColumnPage.java:324)
> 	at org.apache.carbondata.core.datastore.page.encoding.DeltaIntegerCodec.encode(DeltaIntegerCodec.java:71)
> 	at org.apache.carbondata.processing.store.TablePageEncoder.encodeAndCompressMeasures(TablePageEncoder.java:75)
> 	at org.apache.carbondata.processing.store.TablePageEncoder.encode(TablePageEncoder.java:64)
> 	at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(CarbonFactDataHandlerColumnar.java:364)
> 	at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(CarbonFactDataHandlerColumnar.java:65)
> 	at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:749)
> 	at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:726)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message