Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 6BBC7200B85 for ; Thu, 15 Sep 2016 20:45:27 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 6A2AE160AC6; Thu, 15 Sep 2016 18:45:27 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 64EAB160ABA for ; Thu, 15 Sep 2016 20:45:26 +0200 (CEST) Received: (qmail 62948 invoked by uid 500); 15 Sep 2016 18:45:25 -0000 Mailing-List: contact issues-help@carbondata.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@carbondata.incubator.apache.org Delivered-To: mailing list issues@carbondata.incubator.apache.org Received: (qmail 62939 invoked by uid 99); 15 Sep 2016 18:45:25 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 15 Sep 2016 18:45:25 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 38C361A0740 for ; Thu, 15 Sep 2016 18:45:25 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -5.446 X-Spam-Level: X-Spam-Status: No, score=-5.446 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id EItPwWgNq3NT for ; Thu, 15 Sep 2016 18:45:23 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 5A53E60D29 for ; Thu, 15 Sep 2016 18:45:22 +0000 (UTC) Received: (qmail 61522 invoked by uid 99); 15 Sep 2016 18:45:20 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 15 Sep 2016 18:45:20 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id AADDF2C1B75 for ; Thu, 15 Sep 2016 18:45:20 +0000 (UTC) Date: Thu, 15 Sep 2016 18:45:20 +0000 (UTC) From: "Yue Shang (JIRA)" To: issues@carbondata.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (CARBONDATA-243) Filters on dictionary column not always work. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 15 Sep 2016 18:45:27 -0000 Yue Shang created CARBONDATA-243: ------------------------------------ Summary: Filters on dictionary column not always work. Key: CARBONDATA-243 URL: https://issues.apache.org/jira/browse/CARBONDATA-243 Project: CarbonData Issue Type: Bug Components: examples Affects Versions: 0.2.0-incubating Environment: Ubuntu 14.04 Reporter: Yue Shang Priority: Minor Filter on dictionary column not always work. Table loaded contains 100 columns, 100M rows, and one of the column c100's = cardinality is 10M. Dictionary include c100.=20 I tried this query to make sure the value i'm querying is exist. select c100 from big_table4 where c100 like '1234_'=20 =E2=80=8B+-----+ | c100| +-----+ |12340| |12341| |12342| |12343| |12344| |12345| |12346| |12347| |12348| |12349| +-----+ =E2=80=8B But when I tried: " select c100 from big_table4 where c100 like '12345'=E2= =80=8B ", I got a runtime error(Log is at the bottom of this mail). But the most wield part is, some values can be queried while others not. cc.sql("select c100 from big_table4 where c100 like '1000016'") This query will get the exact correct answer. Any idea about this?=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D log =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =E2=80=8BERROR 14-09 17:16:52,024 - [Executor task launch worker-0][partiti= onID:table4;queryID:275884499876481_0]=20 java.lang.NullPointerException =09at org.apache.carbondata.scan.result.iterator.AbstractDetailQueryResultI= terator.intialiseInfos(AbstractDetailQueryResultIterator.java:95) =09at org.apache.carbondata.scan.result.iterator.AbstractDetailQueryResultI= terator.(AbstractDetailQueryResultIterator.java:87) =09at org.apache.carbondata.scan.result.iterator.DetailQueryResultIterator.= (DetailQueryResultIterator.java:47) =09at org.apache.carbondata.scan.executor.impl.DetailQueryExecutor.execute(= DetailQueryExecutor.java:39) =09at org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.(CarbonSc= anRDD.scala:193) =09at org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.s= cala:174) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) =09at org.apache.spark.scheduler.Task.run(Task.scala:88) =09at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.= java:1145) =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor= .java:615) =09at java.lang.Thread.run(Thread.java:745) ERROR 14-09 17:16:52,026 - Exception in task 0.0 in stage 0.0 (TID 0) java.lang.RuntimeException: Exception occurred in query execution.Please ch= eck logs. =09at scala.sys.package$.error(package.scala:27) =09at org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.(CarbonSc= anRDD.scala:203) =09at org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.s= cala:174) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) =09at org.apache.spark.scheduler.Task.run(Task.scala:88) =09at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.= java:1145) =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor= .java:615) =09at java.lang.Thread.run(Thread.java:745) ERROR 14-09 17:16:52,038 - Task 0 in stage 0.0 failed 1 times; aborting job Exception in thread "main" org.apache.spark.SparkException: Job aborted due= to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure:= Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.RuntimeException:= Exception occurred in query execution.Please check logs. =09at scala.sys.package$.error(package.scala:27) =09at org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.(CarbonSc= anRDD.scala:203) =09at org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.s= cala:174) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) =09at org.apache.spark.scheduler.Task.run(Task.scala:88) =09at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.= java:1145) =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor= .java:615) =09at java.lang.Thread.run(Thread.java:745) Driver stacktrace: =09at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DA= GScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1283) =09at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(D= AGScheduler.scala:1271) =09at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(D= AGScheduler.scala:1270) =09at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.= scala:59) =09at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) =09at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala= :1270) =09at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$= 1.apply(DAGScheduler.scala:697) =09at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$= 1.apply(DAGScheduler.scala:697) =09at scala.Option.foreach(Option.scala:236) =09at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGSchedu= ler.scala:697) =09at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(D= AGScheduler.scala:1496) =09at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAG= Scheduler.scala:1458) =09at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAG= Scheduler.scala:1447) =09at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) =09at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:567= ) =09at org.apache.spark.SparkContext.runJob(SparkContext.scala:1824) =09at org.apache.spark.SparkContext.runJob(SparkContext.scala:1837) =09at org.apache.spark.SparkContext.runJob(SparkContext.scala:1850) =09at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:= 215) =09at org.apache.spark.sql.execution.Limit.executeCollect(basicOperators.sc= ala:207) =09at org.apache.spark.sql.DataFrame$$anonfun$collect$1.apply(DataFrame.sca= la:1385) =09at org.apache.spark.sql.DataFrame$$anonfun$collect$1.apply(DataFrame.sca= la:1385) =09at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLEx= ecution.scala:56) =09at org.apache.spark.sql.DataFrame.withNewExecutionId(DataFrame.scala:190= 3) =09at org.apache.spark.sql.DataFrame.collect(DataFrame.scala:1384) =09at org.apache.spark.sql.DataFrame.head(DataFrame.scala:1314) =09at org.apache.spark.sql.DataFrame.take(DataFrame.scala:1377) =09at org.apache.spark.sql.DataFrame.showString(DataFrame.scala:178) =09at org.apache.spark.sql.DataFrame.show(DataFrame.scala:401) =09at org.apache.spark.sql.DataFrame.show(DataFrame.scala:362) =09at org.apache.carbondata.examples.MyBigTest$.main(MyBigTest.scala:107) =09at org.apache.carbondata.examples.MyBigTest.main(MyBigTest.scala) =09at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) =09at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.= java:57) =09at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces= sorImpl.java:43) =09at java.lang.reflect.Method.invoke(Method.java:606) =09at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) Caused by: java.lang.RuntimeException: Exception occurred in query executio= n.Please check logs. =09at scala.sys.package$.error(package.scala:27) =09at org.apache.carbondata.spark.rdd.CarbonScanRDD$$anon$1.(CarbonSc= anRDD.scala:203) =09at org.apache.carbondata.spark.rdd.CarbonScanRDD.compute(CarbonScanRDD.s= cala:174) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:= 38) =09at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:300) =09at org.apache.spark.rdd.RDD.iterator(RDD.scala:264) =09at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) =09at org.apache.spark.scheduler.Task.run(Task.scala:88) =09at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.= java:1145) =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor= .java:615) =09at java.lang.Thread.run(Thread.java:745)=E2=80=8B -- This message was sent by Atlassian JIRA (v6.3.4#6332)