Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B1160200C56 for ; Fri, 14 Apr 2017 09:29:00 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id AF965160B80; Fri, 14 Apr 2017 07:29:00 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D162B160B8C for ; Fri, 14 Apr 2017 09:28:59 +0200 (CEST) Received: (qmail 74793 invoked by uid 500); 14 Apr 2017 07:28:59 -0000 Mailing-List: contact commits-help@carbondata.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@carbondata.incubator.apache.org Delivered-To: mailing list commits@carbondata.incubator.apache.org Received: (qmail 74784 invoked by uid 99); 14 Apr 2017 07:28:59 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Apr 2017 07:28:59 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 8CCFE18F4C2 for ; Fri, 14 Apr 2017 07:28:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.222 X-Spam-Level: X-Spam-Status: No, score=-4.222 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id aMMwCPBBE-oD for ; Fri, 14 Apr 2017 07:28:57 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 37A525FB49 for ; Fri, 14 Apr 2017 07:28:56 +0000 (UTC) Received: (qmail 74740 invoked by uid 99); 14 Apr 2017 07:28:55 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Apr 2017 07:28:55 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 565FCDFC31; Fri, 14 Apr 2017 07:28:55 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: gvramana@apache.org To: commits@carbondata.incubator.apache.org Date: Fri, 14 Apr 2017 07:28:55 -0000 Message-Id: <3ced73daa340492d9205afaf840bbb2c@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/2] incubator-carbondata git commit: fixed NullPointerException for 2nd level subquery archived-at: Fri, 14 Apr 2017 07:29:00 -0000 Repository: incubator-carbondata Updated Branches: refs/heads/master 27a138438 -> f27b4918c fixed NullPointerException for 2nd level subquery Project: http://git-wip-us.apache.org/repos/asf/incubator-carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-carbondata/commit/fc0fc7d9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-carbondata/tree/fc0fc7d9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-carbondata/diff/fc0fc7d9 Branch: refs/heads/master Commit: fc0fc7d906c9f6579bc3fc67b5672cac4de55068 Parents: 27a1384 Author: kunal642 Authored: Tue Apr 11 16:43:39 2017 +0530 Committer: Venkata Ramana G Committed: Fri Apr 14 12:54:09 2017 +0530 ---------------------------------------------------------------------- .../src/test/resources/temp/data1.csv | 4 ++ .../sql/optimizer/CarbonLateDecodeRule.scala | 6 ++- .../carbondata/query/SubQueryTestSuite.scala | 44 ++++++++++++++++++++ 3 files changed, 53 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-carbondata/blob/fc0fc7d9/integration/spark-common-test/src/test/resources/temp/data1.csv ---------------------------------------------------------------------- diff --git a/integration/spark-common-test/src/test/resources/temp/data1.csv b/integration/spark-common-test/src/test/resources/temp/data1.csv new file mode 100644 index 0000000..44e9c10 --- /dev/null +++ b/integration/spark-common-test/src/test/resources/temp/data1.csv @@ -0,0 +1,4 @@ +id,name,rating +1,xyz,5 +2,ghj,2 +3,ghj,3 http://git-wip-us.apache.org/repos/asf/incubator-carbondata/blob/fc0fc7d9/integration/spark2/src/main/scala/org/apache/spark/sql/optimizer/CarbonLateDecodeRule.scala ---------------------------------------------------------------------- diff --git a/integration/spark2/src/main/scala/org/apache/spark/sql/optimizer/CarbonLateDecodeRule.scala b/integration/spark2/src/main/scala/org/apache/spark/sql/optimizer/CarbonLateDecodeRule.scala index 181328d..45cc330 100644 --- a/integration/spark2/src/main/scala/org/apache/spark/sql/optimizer/CarbonLateDecodeRule.scala +++ b/integration/spark2/src/main/scala/org/apache/spark/sql/optimizer/CarbonLateDecodeRule.scala @@ -60,7 +60,11 @@ class CarbonLateDecodeRule extends Rule[LogicalPlan] with PredicateHelper { if (relations.nonEmpty && !isOptimized(plan)) { // In case scalar subquery skip the transformation and update the flag. if (relations.exists(_.carbonRelation.isSubquery.nonEmpty)) { - relations.foreach(p => p.carbonRelation.isSubquery.remove(0)) + relations.foreach{carbonDecoderRelation => + if (carbonDecoderRelation.carbonRelation.isSubquery.nonEmpty) { + carbonDecoderRelation.carbonRelation.isSubquery.remove(0) + } + } LOGGER.info("Skip CarbonOptimizer for scalar/predicate sub query") return plan } http://git-wip-us.apache.org/repos/asf/incubator-carbondata/blob/fc0fc7d9/integration/spark2/src/test/scala/org/apache/spark/carbondata/query/SubQueryTestSuite.scala ---------------------------------------------------------------------- diff --git a/integration/spark2/src/test/scala/org/apache/spark/carbondata/query/SubQueryTestSuite.scala b/integration/spark2/src/test/scala/org/apache/spark/carbondata/query/SubQueryTestSuite.scala new file mode 100644 index 0000000..fbc859f --- /dev/null +++ b/integration/spark2/src/test/scala/org/apache/spark/carbondata/query/SubQueryTestSuite.scala @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.spark.carbondata.query + +import org.apache.spark.sql.Row +import org.apache.spark.sql.common.util.QueryTest +import org.scalatest.BeforeAndAfterAll + +class SubQueryTestSuite extends QueryTest with BeforeAndAfterAll { + + val tempDirPath = s"$resourcesPath/temp" + + override def beforeAll(){ + sql("drop table if exists subquery") + sql("create table subquery(id int, name string, rating float) stored by 'carbondata'") + sql(s"load data local inpath '$tempDirPath/data1.csv' into table subquery") + } + + test("test to check if 2nd level subquery gives correct result") { + checkAnswer(sql( + "select * from subquery where id in(select id from subquery where name in(select name from" + + " subquery where rating=2.0))"), + Seq(Row(2,"ghj",2.0), Row(3,"ghj",3.0))) + } + + override def afterAll() { + sql("drop table if exists subquery") + } +}