carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-222) Query issue for all dimensions are no dictionary columns
Date Thu, 08 Sep 2016 06:52:20 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473004#comment-15473004
] 

ASF GitHub Bot commented on CARBONDATA-222:
-------------------------------------------

Github user kumarvishal09 commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/137#discussion_r77953725
  
    --- Diff: core/src/main/java/org/apache/carbondata/core/carbon/datastore/impl/btree/BTreeDataRefNodeFinder.java
---
    @@ -240,9 +240,16 @@ private int compareIndexes(IndexKey first, IndexKey second) {
                   firstNoDictionaryKeyBuffer.getShort(nonDictionaryKeyOffset + SHORT_SIZE_IN_BYTES);
               secondNodeDictionaryLength =
                   secondNoDictionaryKeyBuffer.getShort(nonDictionaryKeyOffset + SHORT_SIZE_IN_BYTES);
    -          compareResult = ByteUtil.UnsafeComparer.INSTANCE
    -              .compareTo(first.getNoDictionaryKeys(), actualOffset, firstNoDcitionaryLength,
    -                  second.getNoDictionaryKeys(), actualOffset, secondNodeDictionaryLength);
    +          int minLength = Math.min(firstNoDcitionaryLength, secondNodeDictionaryLength);
    --- End diff --
    
    @gvramana Yes you right current logic has some problem we need to maintain offset of both
the keys and when it is not the last key we need to substract the secondNodeDictionaryLength
- actualOffset1 and firstNodeDictionaryLength-actualOffset. 


> Query issue for all dimensions are no dictionary columns
> --------------------------------------------------------
>
>                 Key: CARBONDATA-222
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-222
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Gin-zhj
>            Assignee: Gin-zhj
>            Priority: Minor
>
> step 1:
> CREATE TABLE uniqdata_no (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB
timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10),
DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME,ACTIVE_EMUI_VERSION');
> step 2:
> LOAD DATA INPATH 'D:/download/3lakh_3.csv' into table uniqdata_no OPTIONS('DELIMITER'=','
, 'QUOTECHAR'='"','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> step 3:
> select * from uniqdata_no limit 5;
> the fact file is:
> ,,,,,,,,,,,0
> query failed, catch exception:
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> 	at org.apache.carbondata.core.util.ByteUtil$UnsafeComparer.compareTo(ByteUtil.java:197)
> 	at org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.compareIndexes(BTreeDataRefNodeFinder.java:243)
> 	at org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.findFirstLeafNode(BTreeDataRefNodeFinder.java:121)
> 	at org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.findFirstDataBlock(BTreeDataRefNodeFinder.java:80)
> 	at org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfIndex(CarbonInputFormat.java:546)
> 	at org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:473)
> 	at org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:342)
> 	at org.apache.carbondata.hadoop.CarbonInputFormat.getSplitsNonFilter(CarbonInputFormat.java:304)
> 	at org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:277)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message