carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bhavya Aggarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-999) use carbondata bulket feature,but it doesn't seem to work?
Date Fri, 28 Apr 2017 08:15:04 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988399#comment-15988399
] 

Bhavya Aggarwal commented on CARBONDATA-999:
--------------------------------------------

I tested it with the latest build and it is working fine, it created the two buckets as seen
below, also it is using the ParallelReadMergeSorterWithBucketingImpl only the log has wrong
information because logger for this class is initialized with ParallelReadMergeSorter.class
instead of ParallelReadMergeSorterWithBucketingImpl.class. I have fixed the issue and will
raise a PR for same.

drwxr-xr-x   - hduser supergroup          0 2017-04-28 12:31 /user/hive/warehouse/carbon.store/default/shop_test/Fact
drwxr-xr-x   - hduser supergroup          0 2017-04-28 13:07 /user/hive/warehouse/carbon.store/default/shop_test/Fact/Part0
drwxr-xr-x   - hduser supergroup          0 2017-04-28 12:31 /user/hive/warehouse/carbon.store/default/shop_test/Fact/Part0/Segment_0
-rw-r--r--   1 hduser supergroup        566 2017-04-28 12:31 /user/hive/warehouse/carbon.store/default/shop_test/Fact/Part0/Segment_0/0_batchno0-0-1493362869299.carbonindex
-rw-r--r--   1 hduser supergroup        566 2017-04-28 12:31 /user/hive/warehouse/carbon.store/default/shop_test/Fact/Part0/Segment_0/0_batchno0-1-1493362869299.carbonindex
-rw-r--r--   1 hduser supergroup        824 2017-04-28 12:31 /user/hive/warehouse/carbon.store/default/shop_test/Fact/Part0/Segment_0/part-0-0_batchno0-0-1493362869299.carbondata
-rw-r--r--   1 hduser supergroup        814 2017-04-28 12:31 /user/hive/warehouse/carbon.store/default/shop_test/Fact/Part0/Segment_0/part-0-0_batchno0-1-1493362869299.carbondata


> use carbondata bulket feature,but it doesn't seem to work?
> ----------------------------------------------------------
>
>                 Key: CARBONDATA-999
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-999
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-load
>    Affects Versions: 1.1.0
>         Environment: spark 1.6.2,carbondata 1.1.0 rc1
>            Reporter: xuzhiliang
>
> 1.CREATE TABLE shop_test(platFormId int,sellerNick string,companyGuid STRING,companyName
STRING) STORED BY 'carbondata' TBLPROPERTIES ('BUCKETNUMBER'='2','BUCKETCOLUMNS'='sellerNick')
> 2. .when loading data
> the sorter is type of ParallelReadMergeSorterImpl,not ParallelReadMergeSorterWithBucketingImpl,why
configuration.getBucketingInfo is null?What is wrong with that? Can you fix it?
> 3.hadoop dfs -lsr /Opt/CarbonStore/default/shop_test
> drwxr-xr-x   - root supergroup          0 2017-04-27 15:37 /Opt/CarbonStore/default/shop_test/Fact
> drwxr-xr-x   - root supergroup          0 2017-04-27 15:37 /Opt/CarbonStore/default/shop_test/Fact/Part0
> drwxr-xr-x   - root supergroup          0 2017-04-27 15:37 /Opt/CarbonStore/default/shop_test/Fact/Part0/Segment_0
> -rw-r--r--   3 root supergroup        566 2017-04-27 15:37 /Opt/CarbonStore/default/shop_test/Fact/Part0/Segment_0/0_batchno0-0-1493278648826.carbonindex
> -rw-r--r--   3 root supergroup        891 2017-04-27 15:37 /Opt/CarbonStore/default/shop_test/Fact/Part0/Segment_0/part-0-0_batchno0-0-1493278648826.carbondata



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message