Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 6913B200C7D for ; Tue, 16 May 2017 12:43:13 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 67B0D160BAC; Tue, 16 May 2017 10:43:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 129FA160B9D for ; Tue, 16 May 2017 12:43:11 +0200 (CEST) Received: (qmail 59334 invoked by uid 500); 16 May 2017 10:43:11 -0000 Mailing-List: contact issues-help@carbondata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@carbondata.apache.org Delivered-To: mailing list issues@carbondata.apache.org Received: (qmail 59291 invoked by uid 99); 16 May 2017 10:43:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 May 2017 10:43:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id C64191AFE18 for ; Tue, 16 May 2017 10:43:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id uBA0X0GoIgT5 for ; Tue, 16 May 2017 10:43:05 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 0168261010 for ; Tue, 16 May 2017 10:43:05 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6F386E01A8 for ; Tue, 16 May 2017 10:43:04 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 1FAAA2193A for ; Tue, 16 May 2017 10:43:04 +0000 (UTC) Date: Tue, 16 May 2017 10:43:04 +0000 (UTC) From: "SWATI RAO (JIRA)" To: issues@carbondata.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CARBONDATA-1046) Single_pass_loading is throwing an error in Spark1.6 in automation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 16 May 2017 10:43:13 -0000 [ https://issues.apache.org/jira/browse/CARBONDATA-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012124#comment-16012124 ] SWATI RAO commented on CARBONDATA-1046: --------------------------------------- Above issue was resolved but data load failure occurred now, when executing the query in automation while same query is running fine when we execute it manually. > Single_pass_loading is throwing an error in Spark1.6 in automation > ------------------------------------------------------------------ > > Key: CARBONDATA-1046 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1046 > Project: CarbonData > Issue Type: Bug > Environment: Spark1.6 > Reporter: SWATI RAO > Fix For: 1.1.0 > > Attachments: 7000_UniqData.csv > > > Steps to Reproduce : > Create Table : > CREATE TABLE uniqdata_INCLUDEDICTIONARY (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1'); > +---------+--+ > | Result | > +---------+--+ > +---------+--+ > No rows selected (1.709 seconds) > Load Query : > LOAD DATA INPATH 'hdfs://hadoop-master:54310/BabuStore/Data/uniqdata/7000_UniqData.csv' into table uniqdata_INCLUDEDICTIONARY OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_Pass'='true'); > Stack Trace : > INFO 11-05 13:54:45,047 - Running query 'LOAD DATA INPATH 'hdfs://hadoop-master:54310/BabuStore/Data/uniqdata/7000_UniqData.csv' into table uniqdata_INCLUDEDICTIONARY OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_Pass'='true')' with 44e92bcb-f9e1-4b2e-835e-a82eae525fe4 > INFO 11-05 13:54:45,047 - pool-31-thread-3 Query [LOAD DATA INPATH 'HDFS://HADOOP-MASTER:54310/BABUSTORE/DATA/UNIQDATA/7000_UNIQDATA.CSV' INTO TABLE UNIQDATA_INCLUDEDICTIONARY OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,DOUBLE_COLUMN1,DOUBLE_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='TRUE')] > INFO 11-05 13:54:45,065 - pool-31-thread-3 HDFS lock path:hdfs://192.168.2.145:54310/opt/olapcontent/default/uniqdata_includedictionary/meta.lock > INFO 11-05 13:54:45,097 - Successfully able to get the table metadata file lock > INFO 11-05 13:54:45,099 - pool-31-thread-3 Initiating Direct Load for the Table : (default.uniqdata_includedictionary) > AUDIT 11-05 13:54:45,100 - [hadoop-master][hduser][Thread-150]Data load request has been received for table default.uniqdata_includedictionary > AUDIT 11-05 13:54:45,100 - [hadoop-master][hduser][Thread-150]Data is loading with New Data Flow for table default.uniqdata_includedictionary > ERROR 11-05 13:54:45,104 - Dictionary server Dictionary Server Start Failed > java.net.BindException: Address already in use > at sun.nio.ch.Net.bind0(Native Method) > at sun.nio.ch.Net.bind(Net.java:433) > at sun.nio.ch.Net.bind(Net.java:425) > at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) > at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:125) > at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:485) > at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1089) > at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:430) > at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:415) > at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:903) > at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:198) > at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:348) > at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) > at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > at java.lang.Thread.run(Thread.java:745) > INFO 11-05 13:54:45,131 - pool-31-thread-3 [Block Distribution] > INFO 11-05 13:54:45,131 - pool-31-thread-3 totalInputSpaceConsumed: 1505367 , defaultParallelism: 8 > INFO 11-05 13:54:45,131 - pool-31-thread-3 mapreduce.input.fileinputformat.split.maxsize: 16777216 > INFO 11-05 13:54:45,132 - Total input paths to process : 1 > INFO 11-05 13:54:45,133 - pool-31-thread-3 Executors configured : 1 > INFO 11-05 13:54:45,133 - pool-31-thread-3 Requesting total executors: 1 > INFO 11-05 13:54:45,136 - pool-31-thread-3 Total Time taken to ensure the required executors : 1 > INFO 11-05 13:54:45,136 - pool-31-thread-3 Time elapsed to allocate the required executors: 0 > INFO 11-05 13:54:45,136 - pool-31-thread-3 Total Time taken in block allocation: 3 > INFO 11-05 13:54:45,136 - pool-31-thread-3 Total no of blocks: 1, No.of Nodes: 1 > INFO 11-05 13:54:45,136 - pool-31-thread-3 #Node: hadoop-master no.of.blocks: 1 > INFO 11-05 13:54:45,225 - Block broadcast_2 stored as values in memory (estimated size 122.2 MB, free 227.2 MB) > INFO 11-05 13:54:45,231 - Block broadcast_2_piece0 stored as bytes in memory (estimated size 19.4 KB, free 227.3 MB) > INFO 11-05 13:54:45,231 - Added broadcast_2_piece0 in memory on 192.168.2.145:39344 (size: 19.4 KB, free: 1983.0 MB) > INFO 11-05 13:54:45,232 - Created broadcast 2 from broadcast at NewCarbonDataLoadRDD.scala:108 > INFO 11-05 13:54:45,238 - Starting job: collect at CarbonDataRDDFactory.scala:643 > INFO 11-05 13:54:45,238 - Got job 1 (collect at CarbonDataRDDFactory.scala:643) with 1 output partitions > INFO 11-05 13:54:45,238 - Final stage: ResultStage 1 (collect at CarbonDataRDDFactory.scala:643) > INFO 11-05 13:54:45,238 - Parents of final stage: List() > INFO 11-05 13:54:45,238 - Missing parents: List() > INFO 11-05 13:54:45,239 - Submitting ResultStage 1 (NewCarbonDataLoadRDD[3] at RDD at NewCarbonDataLoadRDD.scala:90), which has no missing parents > INFO 11-05 13:54:45,239 - Preferred Location for split : hadoop-master > INFO 11-05 13:54:45,240 - Block broadcast_3 stored as values in memory (estimated size 11.9 KB, free 227.3 MB) > INFO 11-05 13:54:45,240 - Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.8 KB, free 227.3 MB) > INFO 11-05 13:54:45,241 - Added broadcast_3_piece0 in memory on 192.168.2.145:39344 (size: 5.8 KB, free: 1983.0 MB) > INFO 11-05 13:54:45,241 - Created broadcast 3 from broadcast at DAGScheduler.scala:1006 > INFO 11-05 13:54:45,241 - Submitting 1 missing tasks from ResultStage 1 (NewCarbonDataLoadRDD[3] at RDD at NewCarbonDataLoadRDD.scala:90) > INFO 11-05 13:54:45,241 - Adding task set 1.0 with 1 tasks > INFO 11-05 13:54:45,242 - Starting task 0.0 in stage 1.0 (TID 2, hadoop-master, partition 0,NODE_LOCAL, 2536 bytes) > INFO 11-05 13:54:45,256 - Added broadcast_3_piece0 in memory on hadoop-master:45786 (size: 5.8 KB, free: 7.0 GB) > INFO 11-05 13:54:45,271 - Added broadcast_2_piece0 in memory on hadoop-master:45786 (size: 19.4 KB, free: 7.0 GB) > AUDIT 11-05 13:54:45,290 - [hadoop-master][hduser][Thread-138]Connected io.netty.channel.DefaultChannelHandlerContext@54cef80c > AUDIT 11-05 13:54:46,309 - [hadoop-master][hduser][Thread-139]Connected io.netty.channel.DefaultChannelHandlerContext@295ea64b > ERROR 11-05 13:54:47,339 - nioEventLoopGroup-4-6 > java.lang.IndexOutOfBoundsException: readerIndex(67) + length(25) exceeds writerIndex(80): UnpooledUnsafeDirectByteBuf(ridx: 67, widx: 80, cap: 80) > at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1166) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:676) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:684) > at org.apache.carbondata.core.dictionary.generator.key.DictionaryMessage.readData(DictionaryMessage.java:70) > at org.apache.carbondata.core.dictionary.server.DictionaryServerHandler.channelRead(DictionaryServerHandler.java:59) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > at java.lang.Thread.run(Thread.java:745) > ERROR 11-05 13:54:47,339 - nioEventLoopGroup-4-6 exceptionCaught > java.lang.IndexOutOfBoundsException: readerIndex(67) + length(25) exceeds writerIndex(80): UnpooledUnsafeDirectByteBuf(ridx: 67, widx: 80, cap: 80) > at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1166) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:676) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:684) > at org.apache.carbondata.core.dictionary.generator.key.DictionaryMessage.readData(DictionaryMessage.java:70) > at org.apache.carbondata.core.dictionary.server.DictionaryServerHandler.channelRead(DictionaryServerHandler.java:59) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > at java.lang.Thread.run(Thread.java:745) > WARN 11-05 13:55:59,151 - Lost task 0.1 in stage 0.0 (TID 1, hadoop-master): org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > at org.apache.carbondata.processing.newflow.sort.impl.ParallelReadMergeSorterImpl.checkError(ParallelReadMergeSorterImpl.java:164) > at org.apache.carbondata.processing.newflow.sort.impl.ParallelReadMergeSorterImpl.sort(ParallelReadMergeSorterImpl.java:117) > at org.apache.carbondata.processing.newflow.steps.SortProcessorStepImpl.execute(SortProcessorStepImpl.java:82) > at org.apache.carbondata.processing.newflow.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:92) > at org.apache.carbondata.processing.newflow.DataLoadExecutor.execute(DataLoadExecutor.java:48) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$1.(NewCarbonDataLoadRDD.scala:166) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD.compute(NewCarbonDataLoadRDD.scala:142) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) > at org.apache.spark.scheduler.Task.run(Task.scala:89) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Request timed out for key : DictionaryKey{ columnName='active_emui_version', data='ACTIVE_EMUI_VERSION_20007', dictionaryValue=-1, type=DICT_GENERATION} > at org.apache.carbondata.core.dictionary.client.DictionaryClientHandler.getDictionary(DictionaryClientHandler.java:99) > at org.apache.carbondata.core.dictionary.client.DictionaryClient.getDictionary(DictionaryClient.java:74) > at org.apache.carbondata.processing.newflow.dictionary.DictionaryServerClientDictionary.getOrGenerateKey(DictionaryServerClientDictionary.java:57) > at org.apache.carbondata.processing.newflow.dictionary.DictionaryServerClientDictionary.getOrGenerateKey(DictionaryServerClientDictionary.java:32) > at org.apache.carbondata.processing.newflow.converter.impl.DictionaryFieldConverterImpl.convert(DictionaryFieldConverterImpl.java:115) > at org.apache.carbondata.processing.newflow.converter.impl.RowConverterImpl.convert(RowConverterImpl.java:155) > at org.apache.carbondata.processing.newflow.steps.DataConverterProcessorStepImpl.processRowBatch(DataConverterProcessorStepImpl.java:105) > at org.apache.carbondata.processing.newflow.steps.DataConverterProcessorStepImpl$1.next(DataConverterProcessorStepImpl.java:91) > at org.apache.carbondata.processing.newflow.steps.DataConverterProcessorStepImpl$1.next(DataConverterProcessorStepImpl.java:80) > at org.apache.carbondata.processing.newflow.sort.impl.ParallelReadMergeSorterImpl$SortIteratorThread.call(ParallelReadMergeSorterImpl.java:227) > at org.apache.carbondata.processing.newflow.sort.impl.ParallelReadMergeSorterImpl$SortIteratorThread.call(ParallelReadMergeSorterImpl.java:201) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ... 3 more > Caused by: java.lang.RuntimeException: Request timed out for key : DictionaryKey{ columnName='active_emui_version', data='ACTIVE_EMUI_VERSION_20007', dictionaryValue=-1, type=DICT_GENERATION} > at org.apache.carbondata.core.dictionary.client.DictionaryClientHandler.getDictionary(DictionaryClientHandler.java:94) > ... 14 more > INFO 11-05 13:55:59,152 - Starting task 0.2 in stage 0.0 (TID 3, hadoop-master, partition 0,NODE_LOCAL, 2536 bytes) > AUDIT 11-05 13:55:59,162 - [hadoop-master][hduser][Thread-140]Connected io.netty.channel.DefaultChannelHandlerContext@6d6b8cf6 > AUDIT 11-05 13:56:00,178 - [hadoop-master][hduser][Thread-141]Connected io.netty.channel.DefaultChannelHandlerContext@1f7c81aa > ERROR 11-05 13:56:01,212 - nioEventLoopGroup-4-8 > java.lang.IndexOutOfBoundsException: readerIndex(67) + length(25) exceeds writerIndex(80): UnpooledUnsafeDirectByteBuf(ridx: 67, widx: 80, cap: 80) > at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1166) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:676) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:684) > at org.apache.carbondata.core.dictionary.generator.key.DictionaryMessage.readData(DictionaryMessage.java:70) > at org.apache.carbondata.core.dictionary.server.DictionaryServerHandler.channelRead(DictionaryServerHandler.java:59) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > at java.lang.Thread.run(Thread.java:745) > ERROR 11-05 13:56:01,213 - nioEventLoopGroup-4-8 exceptionCaught > java.lang.IndexOutOfBoundsException: readerIndex(67) + length(25) exceeds writerIndex(80): UnpooledUnsafeDirectByteBuf(ridx: 67, widx: 80, cap: 80) > at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1166) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:676) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:684) > at org.apache.carbondata.core.dictionary.generator.key.DictionaryMessage.readData(DictionaryMessage.java:70) > at org.apache.carbondata.core.dictionary.server.DictionaryServerHandler.channelRead(DictionaryServerHandler.java:59) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > at java.lang.Thread.run(Thread.java:745) > WARN 11-05 13:56:31,750 - Lost task 0.0 in stage 1.0 (TID 2, hadoop-master): org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: > at org.apache.carbondata.processing.newflow.sort.impl.ParallelReadMergeSorterImpl.checkError(ParallelReadMergeSorterImpl.java:164) > at org.apache.carbondata.processing.newflow.sort.impl.ParallelReadMergeSorterImpl.sort(ParallelReadMergeSorterImpl.java:117) > at org.apache.carbondata.processing.newflow.steps.SortProcessorStepImpl.execute(SortProcessorStepImpl.java:82) > at org.apache.carbondata.processing.newflow.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:92) > at org.apache.carbondata.processing.newflow.DataLoadExecutor.execute(DataLoadExecutor.java:48) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$1.(NewCarbonDataLoadRDD.scala:166) > at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD.compute(NewCarbonDataLoadRDD.scala:142) > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) > at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) > at org.apache.spark.scheduler.Task.run(Task.scala:89) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Request timed out for key : DictionaryKey{ columnName='active_emui_version', data='ACTIVE_EMUI_VERSION_20007', dictionaryValue=-1, type=DICT_GENERATION} > at org.apache.carbondata.core.dictionary.client.DictionaryClientHandler.getDictionary(DictionaryClientHandler.java:99) > at org.apache.carbondata.core.dictionary.client.DictionaryClient.getDictionary(DictionaryClient.java:74) > at org.apache.carbondata.processing.newflow.dictionary.DictionaryServerClientDictionary.getOrGenerateKey(DictionaryServerClientDictionary.java:57) > at org.apache.carbondata.processing.newflow.dictionary.DictionaryServerClientDictionary.getOrGenerateKey(DictionaryServerClientDictionary.java:32) > at org.apache.carbondata.processing.newflow.converter.impl.DictionaryFieldConverterImpl.convert(DictionaryFieldConverterImpl.java:115) > at org.apache.carbondata.processing.newflow.converter.impl.RowConverterImpl.convert(RowConverterImpl.java:155) > at org.apache.carbondata.processing.newflow.steps.DataConverterProcessorStepImpl.processRowBatch(DataConverterProcessorStepImpl.java:105) > at org.apache.carbondata.processing.newflow.steps.DataConverterProcessorStepImpl$1.next(DataConverterProcessorStepImpl.java:91) > at org.apache.carbondata.processing.newflow.steps.DataConverterProcessorStepImpl$1.next(DataConverterProcessorStepImpl.java:80) > at org.apache.carbondata.processing.newflow.sort.impl.ParallelReadMergeSorterImpl$SortIteratorThread.call(ParallelReadMergeSorterImpl.java:227) > at org.apache.carbondata.processing.newflow.sort.impl.ParallelReadMergeSorterImpl$SortIteratorThread.call(ParallelReadMergeSorterImpl.java:201) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ... 3 more > Caused by: java.lang.RuntimeException: Request timed out for key : DictionaryKey{ columnName='active_emui_version', data='ACTIVE_EMUI_VERSION_20007', dictionaryValue=-1, type=DICT_GENERATION} > at org.apache.carbondata.core.dictionary.client.DictionaryClientHandler.getDictionary(DictionaryClientHandler.java:94) > ... 14 more > INFO 11-05 13:56:31,750 - Starting task 0.1 in stage 1.0 (TID 4, hadoop-master, partition 0,NODE_LOCAL, 2536 bytes) > AUDIT 11-05 13:56:31,762 - [hadoop-master][hduser][Thread-134]Connected io.netty.channel.DefaultChannelHandlerContext@1e498704 > AUDIT 11-05 13:56:32,777 - [hadoop-master][hduser][Thread-135]Connected io.netty.channel.DefaultChannelHandlerContext@56018951 > ERROR 11-05 13:56:33,800 - nioEventLoopGroup-4-2 > java.lang.IndexOutOfBoundsException: readerIndex(67) + length(25) exceeds writerIndex(80): UnpooledUnsafeDirectByteBuf(ridx: 67, widx: 80, cap: 80) > at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1166) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:676) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:684) > at org.apache.carbondata.core.dictionary.generator.key.DictionaryMessage.readData(DictionaryMessage.java:70) > at org.apache.carbondata.core.dictionary.server.DictionaryServerHandler.channelRead(DictionaryServerHandler.java:59) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > at java.lang.Thread.run(Thread.java:745) > ERROR 11-05 13:56:33,800 - nioEventLoopGroup-4-2 exceptionCaught > java.lang.IndexOutOfBoundsException: readerIndex(67) + length(25) exceeds writerIndex(80): UnpooledUnsafeDirectByteBuf(ridx: 67, widx: 80, cap: 80) > at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1166) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:676) > at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:684) > at org.apache.carbondata.core.dictionary.generator.key.DictionaryMessage.readData(DictionaryMessage.java:70) > at org.apache.carbondata.core.dictionary.server.DictionaryServerHandler.channelRead(DictionaryServerHandler.java:59) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) > at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.15#6346)