From derby-user-return-10107-apmail-db-derby-user-archive=db.apache.org@db.apache.org Wed Nov 12 16:12:48 2008 Return-Path: Delivered-To: apmail-db-derby-user-archive@www.apache.org Received: (qmail 50214 invoked from network); 12 Nov 2008 16:12:48 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 12 Nov 2008 16:12:48 -0000 Received: (qmail 49371 invoked by uid 500); 12 Nov 2008 16:12:54 -0000 Delivered-To: apmail-db-derby-user-archive@db.apache.org Received: (qmail 49349 invoked by uid 500); 12 Nov 2008 16:12:54 -0000 Mailing-List: contact derby-user-help@db.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: Reply-To: "Derby Discussion" Delivered-To: mailing list derby-user@db.apache.org Received: (qmail 49337 invoked by uid 99); 12 Nov 2008 16:12:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Nov 2008 08:12:54 -0800 X-ASF-Spam-Status: No, hits=-1.4 required=10.0 tests=RCVD_IN_DNSWL_MED,RCVD_NUMERIC_HELO,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [192.18.6.24] (HELO gmp-eb-inf-2.sun.com) (192.18.6.24) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 12 Nov 2008 16:11:33 +0000 Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mACGC9ga024022 for ; Wed, 12 Nov 2008 16:12:09 GMT Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KA800E019H4XI00@fe-emea-10.sun.com> (original mail from Kristian.Waagan@Sun.COM) for derby-user@db.apache.org; Wed, 12 Nov 2008 16:12:09 +0000 (GMT) Received: from 77.16.141.186.tmi.telenormobil.no ([77.16.141.186]) by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KA8001AWABXUFI0@fe-emea-10.sun.com> for derby-user@db.apache.org; Wed, 12 Nov 2008 16:12:02 +0000 (GMT) Date: Wed, 12 Nov 2008 17:11:56 +0100 From: Kristian Waagan Subject: Re: Using derby to parse an SQL statement In-reply-to: Sender: Kristian.Waagan@Sun.COM To: Derby Discussion Message-id: <491B004C.8030402@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-15 Content-transfer-encoding: 8BIT References: <491AF2B7.6080600@sun.com> User-Agent: Thunderbird 2.0.0.17 (Macintosh/20080914) X-Virus-Checked: Checked by ClamAV on apache.org Christian Riedel wrote: > Hi Rick, > > first of all thanks for your answer ... now the relations have become > a lot clearer ... > > Your are right, there is a lot of things to be done that we probably > don't want to go through. You asked why we cannot take the whole derby > engine and use it ... well there is nor real reason not to do so. The > only "problem" I see is, that derby is a dbms - if I am not mistaken - > and we only have an SQL statement that we extract from a text file and > want tot analyze it to extract some metadata from it. > > So if we take the derby engine as it is, how can I prevent that we > have to set up a "dummy" DB in order to be able to actually use thje > parsing feature .... Hello, Note sure if it helps, but Derby does offer a storeless engine. I don't know if you can extract the AST with a little hacking, but it might be worth further investigation. Also, I don't know at which time it throws the exception about missing tables etc. Just as an example, you can write things like "values 1+4", but as if you write "select * from a", you get an error message saying a doesn't exist. I don't know what the internal state of Derby looks like at this point. regards, -- Kristian > > I hope you see my point. > > We could live with setting up a dummy DB ... and I do think that the > derby AST offers all information we need. It's just that I don't see > how we can set this thing up. So having a dummy DB is necessary to be > able to intercept the parsing process to get hold of the AST? Can we > actually access the AST if we choose to set up a dummy DB? I think > that would be something we could live with ;-) > > > Thanks for your support > > Christian > > > Rick Hillegas schrieb: >> Hi Christian, >> >> I think you will have difficulty isolating the Parser from the rest >> of the SQL interpreter. In theory, you should be able to isolate the >> compiler from the execution engine and the storage layer--but that is >> an untested theory. >> >> The Parser wants to turn out abstract syntax trees (AST). Ideally, >> the Parser would just need to ask a NodeFactory for AST nodes and you >> could supply your own NodeFactory. But I think that there is a fair >> amount of coupling between the Parser and Derby's concrete >> implementation of NodeFactory. I think that you could uncouple the >> two, but you may not want to spend your time on that. >> >> So the Parser is going to force you to pull in the AST nodes. Once >> you do that, you will end up with the whole compiler. In particular, >> the AST nodes (and the Parser itself) expect that you will supply an >> implementation of LanguageConnectionContext, the master state >> variable for the whole SQL interpreter. Untangling that requirement >> is another chunk of work you may not want to do. >> >> Then there is the Monitor. It has been a while since I was in that >> code but I seem to recall that fairly early on the Monitor wants to >> fault in a storage layer. In theory you ought to be able to supply >> the Monitor a list of modules that doesn't include a storage layer. >> But since no-one runs in this configuration, there are probably a lot >> of undocumented surprises that you may not want to fix either. >> >> Can I ask you what breaks if you just pull in the whole Derby engine? >> Are you concerned that you will fault in too much code that you >> barely use? Are you concerned that you'll end up with a dummy >> database that you don't need? Are Derby's AST nodes not a usable >> representation of statement syntax? >> >> Thanks, >> -Rick >> >> Christian Riedel wrote: >>> Hi there, >>> >>> we are working on a small project where we need to analyze an SQL >>> statement that can be of any kind: very simple, with inner selects, >>> complex join etc. >>> >>> We figured it inappropriate to start to write our own parser when >>> there are other projects, like derby, out there that can do it much >>> better than we would possibly do ... so this was our idea: >>> >>> Can we use derby to create an instance of Parser >>> (org.apache.derby.iapi.sql.compile.Parser.class) and let our SQL >>> statement be parsed by calling the parse() method on this instance? >>> What we want to have is a syntax tree of the statement that allows >>> us to see which tables and which fields are accessed / included in >>> the statement (including any possibly done "renames" á la SELECT >>> street AS "ADDRESS" FROM USER_DATA ). >>> >>> The problem is, that we are stuck ... we spent several days now to >>> try to find the proper way to create an instance of the Parser. Is >>> it possible at all without having to set up a running derby system? >>> >>> Is the Monitor class the right entry point? How can we create a >>> CompilerContext so that a Parser instance can be created? >>> >>> >>> This sure is off-topic but we don't see any way through all this. >>> Can you help us? >>> >>> >>> Thanks in advance >>> >>> Christian >>> >> >> >