Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 08B0118A0D for ; Tue, 29 Dec 2015 20:14:07 +0000 (UTC) Received: (qmail 79364 invoked by uid 500); 29 Dec 2015 20:14:05 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 79282 invoked by uid 500); 29 Dec 2015 20:14:05 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 79272 invoked by uid 99); 29 Dec 2015 20:14:05 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Dec 2015 20:14:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id B954F1A011C for ; Tue, 29 Dec 2015 20:14:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id OXpHzsYkNeyq for ; Tue, 29 Dec 2015 20:14:04 +0000 (UTC) Received: from mail-lf0-f50.google.com (mail-lf0-f50.google.com [209.85.215.50]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 90BFB439B4 for ; Tue, 29 Dec 2015 20:14:03 +0000 (UTC) Received: by mail-lf0-f50.google.com with SMTP id p203so219386662lfa.0 for ; Tue, 29 Dec 2015 12:14:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Rg5BxqAgyWxe1ZHXrTfDDT2HWPprHQwE8jib3olrQ88=; b=N1U3KqqNiQmNas7nGYFBX7SfFgno/ux7asT91Ol4DBWZqhUwohNSsG1QoRffTjzf+X HtawWqrKpY6zSq5hZzNKGJ7VWegN4dBp8IYOZKO0nSRC+LHMX0HHchEJwyGwx3zV5Hqh wSqDRDGq/96r3r7s2OKqgemqpvz5yYr4pkJGEec1jayuDquLRgcfr0wIjWkwkg9zJb32 SpkyCfzew5+ZDhocczdp9iSIwwRuKGf+MuqiU8d1/jE/uH0+yBbm4M/O6g0g9gv4KXoa RM1FIkXH91LRBjzA/BX9feh56726pCUh8KvTsv0KJs1AlmtOHsdy3d2u8tL4/mdH/TAt mK0A== MIME-Version: 1.0 X-Received: by 10.25.159.9 with SMTP id i9mr14842221lfe.109.1451420042201; Tue, 29 Dec 2015 12:14:02 -0800 (PST) Received: by 10.25.163.200 with HTTP; Tue, 29 Dec 2015 12:14:02 -0800 (PST) In-Reply-To: References: Date: Tue, 29 Dec 2015 15:14:02 -0500 Message-ID: Subject: Re: hacking the hive ql parser? From: Edward Capriolo To: "user@hive.apache.org" Content-Type: multipart/alternative; boundary=001a1141198009153605280f104d --001a1141198009153605280f104d Content-Type: text/plain; charset=UTF-8 hive --service lineage 'hql' exists i believe. On Tue, Dec 29, 2015 at 3:05 PM, Yang wrote: > I'm trying to create a utility to parse out the data lineage (i.e. DAG > dependency graph) among all my hive scripts. > > to do this I need to parse out the input and output tables from a query. > does this ability existing already? if not, I'm going to hack the parser. > I am not very familiar with the parser code structure of hive, could > anybody give me some tips on where to start? > (I see the .g files, but not sure where is the rest ---- I am more > familiar with the ASTvisitor paradigm in antlr, but can't find similar > files in the parser dir) > > > thanks > Yang > --001a1141198009153605280f104d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
hive --service lineage 'hql' exists i believe.

On Tue, Dec 29,= 2015 at 3:05 PM, Yang <teddyyyy123@gmail.com> wrote:
I'm trying to create a= utility to parse out the data lineage (i.e. DAG dependency graph) among al= l my hive scripts.

to do this I need to parse out the in= put and output tables from a query.=C2=A0
does this ability exist= ing already? if not, I'm going to hack the parser.
I am not v= ery familiar with the parser code structure of hive, could anybody give me = some tips on where to start?
(I see the .g files, but not sure wh= ere is the rest ---- I am more familiar with the ASTvisitor paradigm in ant= lr, but can't find similar files in the parser dir)


thanks
Yang

--001a1141198009153605280f104d--