From dev-return-370-apmail-optiq-dev-archive=optiq.apache.org@optiq.incubator.apache.org Sat Sep 20 19:00:48 2014 Return-Path: X-Original-To: apmail-optiq-dev-archive@minotaur.apache.org Delivered-To: apmail-optiq-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B9E4511EF4 for ; Sat, 20 Sep 2014 19:00:48 +0000 (UTC) Received: (qmail 76704 invoked by uid 500); 20 Sep 2014 19:00:48 -0000 Delivered-To: apmail-optiq-dev-archive@optiq.apache.org Received: (qmail 76664 invoked by uid 500); 20 Sep 2014 19:00:48 -0000 Mailing-List: contact dev-help@optiq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@optiq.incubator.apache.org Delivered-To: mailing list dev@optiq.incubator.apache.org Received: (qmail 76652 invoked by uid 99); 20 Sep 2014 19:00:47 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 20 Sep 2014 19:00:47 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of julianhyde@gmail.com designates 209.85.192.174 as permitted sender) Received: from [209.85.192.174] (HELO mail-pd0-f174.google.com) (209.85.192.174) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 20 Sep 2014 19:00:21 +0000 Received: by mail-pd0-f174.google.com with SMTP id g10so1439943pdj.5 for ; Sat, 20 Sep 2014 12:00:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=en0jx98wDKIiLGnK2TUx2tvcUvvsMUSP3ouxWyBRZf4=; b=z5A7U1Atz11irHxfUMgakoLFMNBCJFjfI+o3dC6O6Jw0hWThhVUmgBiYGIY+t0kFtL 29igJoOBFlYKCjtbCPzEL+d6Ib5buE6BY4qt3k8YQCHXp6suGT3jvXzT1SB9tEyQWcrP IMzRv4cLLvAw5iAflEpE0lyeJP+Pn1JT7Fd6FejO6NGa0Ik0tQswKsmNDOZpYbUrjWw+ /ZLLOm+1Kuj6Inf/TsPMQI414N+JXFUL+edSKRKTCijL/a1IrrYYTeqfZdytw9mp/Ymz 0o27uL/0ZVxdbXs0KSUj3cisHXIGFzGFz+9I1FHewbMCdKm+OZhE+MBj1KZ5mZhDFn8H nMuQ== X-Received: by 10.66.229.8 with SMTP id sm8mr12693562pac.66.1411239620160; Sat, 20 Sep 2014 12:00:20 -0700 (PDT) Received: from [192.168.2.109] (c-71-198-186-231.hsd1.ca.comcast.net. [71.198.186.231]) by mx.google.com with ESMTPSA id d3sm5184333pbu.18.2014.09.20.12.00.19 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 20 Sep 2014 12:00:19 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Filter Push down - passing RexCall from EnumerableRel implement method. From: Julian Hyde In-Reply-To: Date: Sat, 20 Sep 2014 12:00:18 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <5A9D8394-B312-4AEA-A55D-C52BB0A18F0F@gmail.com> References: To: dev@optiq.incubator.apache.org X-Mailer: Apple Mail (2.1878.6) X-Virus-Checked: Checked by ClamAV on apache.org Good question. A similar question came up during the hackathon. The current code generation converts everything to strings. You can=92t = pass an object through =97 you need to generate java code that will = create an identical object. That often means serializing the object to = something like a JSON string and generating a call to de-serialize it. It=92s a bit painful, but it does mean that you can evaluate that code = in another JVM (even on another machine). Note that RelJsonWriter can serialize RelNodes to JSON, and = RelJsonReader can read them back. Take a look at RelWriterTest and you = will see that there is a way to convert RexNodes to JSON also. (It=92s = not very easy=85 you could contribute some library methods rexToJson and = jsonToRex when you=92ve figured out how RelJsonWriter and RelJsonReader = do it.) There is a back door which is DataContext. Generated code assumes that = there is a variable =93DataContext root=94. That is the entry point for = all objects (e.g. schemas and open connections) when the plan is = running. Generated code can call DataContext.get(name) to get an object = that has been stashed in the context. So, you could stash your RexCall = in DataContext with a unique name, and generate code that calls = something like (RexCall) root.get(=93name$123=94) In summary, it=92s not very easy but it=92s possible. Let us know how = you get on. Julian On Sep 17, 2014, at 9:14 AM, Abishek Baskaran = wrote: > Hi, > I need to push down the filter expression to my custom data source. I = see > that Optiq represents the filter expression as a RexCall and I wish to = pass > the RexCall object to my custom table. I am able to pass Strings and > Arrays, but unable to figure out how to pass RexCall. >=20 > For example, in this codebase > = > , > We have the implement method that's called on successful push down = rule. >=20 > public Result implement(EnumerableRelImplementor implementor, Prefer = pref) { > PhysType physType =3D > PhysTypeImpl.of( > implementor.getTypeFactory(), > getRowType(), > pref.preferArray()); >=20 > if (table instanceof JsonTable) { > return implementor.result( > physType, > Blocks.toBlock( > Expressions.call(table.getExpression(JsonTable.class), > "enumerable"))); > } > return implementor.result( > physType, > Blocks.toBlock( > Expressions.call(table.getExpression(CsvTable.class), = "project", > Expressions.constant(fields)))); > } >=20 > Expressions.constant(fields) is good for Strings/Arrays. =46rom Splunk > adapter I also figured out how to pass Lists. My requirement is to = pass a > RexCall object, would appreciate any guidance here. >=20 > Thanks, > Abishek