carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neha Bhardwaj (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-1414) Show Segments raises exception for a Partition Table after Updation.
Date Tue, 05 Sep 2017 07:33:00 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153216#comment-16153216
] 

Neha Bhardwaj commented on CARBONDATA-1414:
-------------------------------------------

Hi [~chenliang613] , The above query is raising the exception at COMPACTION now.

0: jdbc:hive2://localhost:10000> ALTER TABLE list_partition_table_string COMPACT 'Minor';
Error: java.lang.RuntimeException: Compaction failed. Please check logs for more info. Exception
in compaction org.apache.spark.SparkException: Job aborted due to stage failure: Task 2 in
stage 33.0 failed 1 times, most recent failure: Lost task 2.0 in stage 33.0 (TID 149, localhost,
executor driver): java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:653)
	at java.util.ArrayList.get(ArrayList.java:429)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.assignComplexOrdinal(SegmentProperties.java:472)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.fillDimensionAndMeasureDetails(SegmentProperties.java:397)
	at org.apache.carbondata.core.datastore.block.SegmentProperties.<init>(SegmentProperties.java:173)
	at org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:161)
	at org.apache.carbondata.spark.rdd.CarbonMergerRDD.internalCompute(CarbonMergerRDD.scala:79)
	at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:62)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace: (state=,code=0)

I think it is because of Complex Datatype in the table schema.

For all other scenarios, the fix is working.

> Show Segments raises exception for a Partition Table after Updation.
> --------------------------------------------------------------------
>
>                 Key: CARBONDATA-1414
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1414
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-query
>    Affects Versions: 1.2.0
>         Environment: spark 2.1
>            Reporter: Neha Bhardwaj
>            Assignee: Liang Chen
>         Attachments: list_partition_table.csv
>
>
> 1. Create Partition Table :
> DROP TABLE IF EXISTS list_partition_table_string;
>  CREATE TABLE list_partition_table_string(shortField SHORT, intField INT, bigintField
LONG, doubleField DOUBLE, timestampField TIMESTAMP, decimalField DECIMAL(18,2), dateField
DATE, charField CHAR(5), floatField FLOAT, complexData ARRAY<STRING> ) PARTITIONED BY
(stringField STRING) STORED BY 'carbondata' TBLPROPERTIES('PARTITION_TYPE'='LIST', 'LIST_INFO'='Asia,
America, Europe', 'DICTIONARY_EXCLUDE'='stringfield');
> 2. Load Data :
> load data inpath 'hdfs://localhost:54310/CSV/list_partition_table.csv' into table list_partition_table_string
options('FILEHEADER'='shortfield,intfield,bigintfield,doublefield,stringfield,timestampfield,decimalfield,datefield,charfield,floatfield,complexdata',
'COMPLEX_DELIMITER_LEVEL_1'='$','COMPLEX_DELIMITER_LEVEL_2'='#', 'SINGLE_PASS'='TRUE');
> 3. Update Data :
> update list_partition_table_string set (stringfield)=('China') where stringfield = 'Japan'
;
>  update list_partition_table_string set (stringfield)=('Japan') where stringfield >
'Europe' ;
>  update list_partition_table_string set (stringfield)=('Asia') where stringfield <
'Europe' ;
> 4. Compaction :
> ALTER TABLE list_partition_table_string COMPACT 'Minor';
>  Show segments for table list_partition_table_string;
> Expected Output: Segments Must be Displayed.
> Actual Output : Error: java.lang.NullPointerException (state=,code=0)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message