Mailing-List: contact issues-help@carbondata.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@carbondata.apache.org
Date: Wed, 22 Nov 2017 12:43:00 +0000 (UTC)
From: "anubhav tarar (JIRA)" <jira@apache.org>
To: issues@carbondata.apache.org
Message-ID: <JIRA.13120218.1511354556000.298960.1511354580334@Atlassian.JIRA>
In-Reply-To: <JIRA.13120218.1511354556000@Atlassian.JIRA>
References: <JIRA.13120218.1511354556000@Atlassian.JIRA> <JIRA.13120218.1511354556959@jira-lw-us.apache.org>
Subject: [jira] [Created] (CARBONDATA-1798) class cast exception when
 loading data from a hive table to carbondata table
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 22 Nov 2017 12:43:06 -0000

anubhav tarar created CARBONDATA-1798:
-----------------------------------------

             Summary: class cast exception when loading data from a hive table to carbondata table
                 Key: CARBONDATA-1798
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1798
             Project: CarbonData
          Issue Type: Bug
          Components: spark-integration
    Affects Versions: 1.3.0
         Environment: spark2.1
            Reporter: anubhav tarar
            Assignee: anubhav tarar


1. i created the hive tpch table named lineitem with around 10 gb data

CREATE TABLE LINEITEM ( L_ORDERKEY INT ,L_PARTKEY INT ,L_SUPPKEY INT ,L_LINENUMBER INT ,L_QUANTITY DECIMAL(15,2)  ,L_EXTENDEDPRICE DECIMAL(15,2)  ,L_DISCOUNT DECIMAL(15,2)  ,L_TAX DECIMAL(15,2)  ,L_RETURNFLAG STRING ,L_LINESTATUS STRING ,L_SHIPDATE Date ,L_COMMITDATE Date ,L_RECEIPTDATE Date ,L_SHIPINSTRUCT STRING ,L_SHIPMODE STRING ,L_COMMENT STRING) stored as orc;

2.tried to insert the data from that hive table into carbondata table 

CREATE TABLE LINEITEM ( L_ORDERKEY INT ,L_PARTKEY INT ,L_SUPPKEY INT ,L_LINENUMBER INT ,L_QUANTITY DECIMAL(15,2)  ,L_EXTENDEDPRICE DECIMAL(15,2)  ,L_DISCOUNT DECIMAL(15,2)  ,L_TAX DECIMAL(15,2)  ,L_RETURNFLAG STRING ,L_LINESTATUS STRING ,L_SHIPDATE Date ,L_COMMITDATE Date ,L_RECEIPTDATE Date ,L_SHIPINSTRUCT STRING ,L_SHIPMODE STRING ,L_COMMENT STRING) stored by 'carbondata';

then i used INSERT INTO LINEITEM SELECT * FROM LINEITEM_ORC;


here are error logs

17/11/22 16:56:40 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on slave1:41401 (size: 25.2 KB, free: 3.0 GB)
17/11/22 16:59:12 WARN TaskSetManager: Lost task 0.0 in stage 15.0 (TID 20, hadoop-master, executor 2): org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: 
	at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:114)
	at org.apache.carbondata.processing.loading.steps.SortProcessorStepImpl.execute(SortProcessorStepImpl.java:62)
	at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:87)
	at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:50)
	at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD$$anon$2.<init>(NewCarbonDataLoadRDD.scala:391)
	at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD.internalCompute(NewCarbonDataLoadRDD.scala:354)
	at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:60)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException: 
	at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.checkForFailure(UnsafeIntermediateMerger.java:188)
	at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.finish(UnsafeIntermediateMerger.java:171)
	at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:107)
	... 14 more
Caused by: java.util.concurrent.ExecutionException: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException: 
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.checkForFailure(UnsafeIntermediateMerger.java:185)
	... 16 more
Caused by: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException: 
	at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:142)
	at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:45)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 3 more
Caused by: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to [B
	at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.writeDataTofile(UnsafeIntermediateFileMerger.java:333)
	at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:113)
	... 5 more

17/11/22 16:59:12 INFO TaskSetManager: Starting task 0.1 in stage 15.0 (TID 22, hadoop-master, executor 2, partition 0, NODE_LOCAL, 7696 bytes)


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)