Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E939F200BFE for ; Mon, 16 Jan 2017 12:52:33 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id E7DAC160B30; Mon, 16 Jan 2017 11:52:33 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E5E17160B22 for ; Mon, 16 Jan 2017 12:52:32 +0100 (CET) Received: (qmail 24751 invoked by uid 500); 16 Jan 2017 11:52:32 -0000 Mailing-List: contact issues-help@carbondata.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@carbondata.incubator.apache.org Delivered-To: mailing list issues@carbondata.incubator.apache.org Received: (qmail 24742 invoked by uid 99); 16 Jan 2017 11:52:32 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Jan 2017 11:52:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id AA7F5C204D for ; Mon, 16 Jan 2017 11:52:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id VJP6n1_XO19J for ; Mon, 16 Jan 2017 11:52:30 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 622D15F4A7 for ; Mon, 16 Jan 2017 11:52:29 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 26375E865F for ; Mon, 16 Jan 2017 11:52:28 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 81FD125288 for ; Mon, 16 Jan 2017 11:52:26 +0000 (UTC) Date: Mon, 16 Jan 2017 11:52:26 +0000 (UTC) From: "Ravindra Pesala (JIRA)" To: issues@carbondata.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CARBONDATA-643) When we are passing ALL_DICTIONARY_PATH' in load query ,it is throwing null pointer exception. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 16 Jan 2017 11:52:34 -0000 [ https://issues.apache.org/jira/browse/CARBONDATA-643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-643: --------------------------------------- Assignee: Manohar Vanam > When we are passing ALL_DICTIONARY_PATH' in load query ,it is throwing null pointer exception. > ---------------------------------------------------------------------------------------------- > > Key: CARBONDATA-643 > URL: https://issues.apache.org/jira/browse/CARBONDATA-643 > Project: CarbonData > Issue Type: Bug > Components: data-load > Affects Versions: 1.0.0-incubating > Reporter: Payal > Assignee: Manohar Vanam > Priority: Minor > Attachments: 7000_UniqData.csv > > > When we are passing ALL_DICTIONARY_PATH' in load query ,it is throwing null pointer exception. > //CREATE TABLE > CREATE TABLE uniq_include_dictionary (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,Double_COLUMN2,DECIMAL_COLUMN2'); > //LOAD QUERY > LOAD DATA INPATH 'hdfs://localhost:54311/payal/7000_UniqData.csv' into table uniq_include_dictionary OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='""','BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='false','ALL_DICTIONARY_PATH'='hdfs://localhost:54311/opt/alldictionary/data.dictionary'); > Error: java.lang.NullPointerException (state=,code=0) > LOGS > INFO 16-01 16:56:37,624 - Running query 'LOAD DATA INPATH 'hdfs://localhost:54311/payal/7000_UniqData.csv' into table uniq_include_dictionary OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='false','MULTILINE'='true','ALL_DICTIONARY_PATH'='hdfs://localhost:54311/opt/alldictionary/data.dictionary')' with 17ec3816-91e3-462e-aa9b-a7ae76405564 > INFO 16-01 16:56:37,625 - pool-26-thread-34 Query [LOAD DATA INPATH 'HDFS://LOCALHOST:54311/PAYAL/7000_UNIQDATA.CSV' INTO TABLE UNIQ_INCLUDE_DICTIONARY OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_LOGGER_ENABLE'='TRUE', 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,DOUBLE_COLUMN1,DOUBLE_COLUMN2,INTEGER_COLUMN1','SINGLE_PASS'='FALSE','MULTILINE'='TRUE','ALL_DICTIONARY_PATH'='HDFS://LOCALHOST:54311/OPT/ALLDICTIONARY/DATA.DICTIONARY')] > INFO 16-01 16:56:37,641 - Successfully able to get the table metadata file lock > INFO 16-01 16:56:37,644 - pool-26-thread-34 Initiating Direct Load for the Table : (default.uniq_include_dictionary) > INFO 16-01 16:56:37,644 - pool-26-thread-34 Generate global dictionary from dictionary files! > ERROR 16-01 16:56:37,645 - pool-26-thread-34 Exception occured:File does not exist: hdfs://localhost:54311/opt/alldictionary/data.dictionary > ERROR 16-01 16:56:37,645 - pool-26-thread-34 generate global dictionary failed > java.lang.NullPointerException > at org.apache.carbondata.core.datastorage.store.filesystem.AbstractDFSCarbonFile.getName(AbstractDFSCarbonFile.java:83) > at org.apache.carbondata.spark.util.GlobalDictionaryUtil$.validateAllDictionaryPath(GlobalDictionaryUtil.scala:649) > at org.apache.carbondata.spark.util.GlobalDictionaryUtil$.generateGlobalDictionary(GlobalDictionaryUtil.scala:743) > at org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:569) > at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58) > at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56) > at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:70) > at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132) > at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130) > at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) > at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130) > at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:55) > at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:55) > at org.apache.spark.sql.DataFrame.(DataFrame.scala:145) > at org.apache.spark.sql.DataFrame.(DataFrame.scala:130) > at org.apache.spark.sql.CarbonContext.sql(CarbonContext.scala:139) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:211) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:154) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:151) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:164) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > ERROR 16-01 16:56:37,645 - pool-26-thread-34 > java.lang.NullPointerException > at org.apache.carbondata.core.datastorage.store.filesystem.AbstractDFSCarbonFile.getName(AbstractDFSCarbonFile.java:83) > at org.apache.carbondata.spark.util.GlobalDictionaryUtil$.validateAllDictionaryPath(GlobalDictionaryUtil.scala:649) > at org.apache.carbondata.spark.util.GlobalDictionaryUtil$.generateGlobalDictionary(GlobalDictionaryUtil.scala:743) > at org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:569) > at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58) > at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56) > at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:70) > at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132) > at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130) > at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) > at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130) > at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:55) > at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:55) > at org.apache.spark.sql.DataFrame.(DataFrame.scala:145) > at org.apache.spark.sql.DataFrame.(DataFrame.scala:130) > at org.apache.spark.sql.CarbonContext.sql(CarbonContext.scala:139) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:211) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:154) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:151) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:164) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > AUDIT 16-01 16:56:37,645 - [sid-vostro-3546][hduser][Thread-263]Dataload failure for default.uniq_include_dictionary. Please check the logs > INFO 16-01 16:56:37,646 - pool-26-thread-34 Successfully deleted the lock file /tmp/default/uniq_include_dictionary/meta.lock > INFO 16-01 16:56:37,646 - Table MetaData Unlocked Successfully after data load > ERROR 16-01 16:56:37,646 - Error executing query, currentState RUNNING, > java.lang.NullPointerException > at org.apache.carbondata.core.datastorage.store.filesystem.AbstractDFSCarbonFile.getName(AbstractDFSCarbonFile.java:83) > at org.apache.carbondata.spark.util.GlobalDictionaryUtil$.validateAllDictionaryPath(GlobalDictionaryUtil.scala:649) > at org.apache.carbondata.spark.util.GlobalDictionaryUtil$.generateGlobalDictionary(GlobalDictionaryUtil.scala:743) > at org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:569) > at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58) > at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56) > at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:70) > at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132) > at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130) > at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) > at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130) > at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:55) > at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:55) > at org.apache.spark.sql.DataFrame.(DataFrame.scala:145) > at org.apache.spark.sql.DataFrame.(DataFrame.scala:130) > at org.apache.spark.sql.CarbonContext.sql(CarbonContext.scala:139) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:211) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:154) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:151) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:164) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > ERROR 16-01 16:56:37,646 - Error running hive query: > org.apache.hive.service.cli.HiveSQLException: java.lang.NullPointerException > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:246) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:154) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:151) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:164) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)