Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 258D618BB9 for ; Wed, 30 Dec 2015 05:45:55 +0000 (UTC) Received: (qmail 53199 invoked by uid 500); 30 Dec 2015 05:45:53 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 53130 invoked by uid 500); 30 Dec 2015 05:45:53 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 53120 invoked by uid 99); 30 Dec 2015 05:45:53 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Dec 2015 05:45:53 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id DB3861A0187 for ; Wed, 30 Dec 2015 05:45:52 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.755 X-Spam-Level: *** X-Spam-Status: No, score=3.755 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HK_RANDOM_ENVFROM=0.626, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 52CP4LutBrbV for ; Wed, 30 Dec 2015 05:45:51 +0000 (UTC) Received: from mail-ig0-f182.google.com (mail-ig0-f182.google.com [209.85.213.182]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 244C3215DD for ; Wed, 30 Dec 2015 05:45:51 +0000 (UTC) Received: by mail-ig0-f182.google.com with SMTP id to4so174006331igc.0 for ; Tue, 29 Dec 2015 21:45:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=aGqlzyst5WpC1hCWJz8+QVmsJsRquAEPbyCuP0c3uII=; b=mDc5TV7xA4OcA7DysHqmBEQXhdTl71tmdjeXHbNsbTwbagGFyE8zPuV47fh8jY+WJC XyYXtw6DsD0YC+h+7v/RoHMekd9Vh6nKxRbrSlgOq2cDVGDSVvGpmglA449R00OTxxmR pegM1fRX/yXhW6Iw3YIq53ZteIz1u0VPIewL7zt77NyBfHUsSKTJ94602tULj8eDh2U4 DGgbs7NEreRDxSH4RL+sQceQ27nJVTBfEZ9s/eTQuN3ddnBhRMq0ZSm65+DduliPny/c J8Rvs5QX47PRiq5ItG388fDQoHEJyMEmEyEBMPc6izv2GLNW2n+p9jdjXqvL/FZUhE2r 7ryA== X-Received: by 10.50.92.99 with SMTP id cl3mr53907082igb.71.1451454344412; Tue, 29 Dec 2015 21:45:44 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.162.199 with HTTP; Tue, 29 Dec 2015 21:45:24 -0800 (PST) In-Reply-To: References: From: Yang Date: Tue, 29 Dec 2015 21:45:24 -0800 Message-ID: Subject: Re: hacking the hive ql parser? To: user@hive.apache.org Content-Type: multipart/alternative; boundary=047d7b10cf2d9b440c0528170c4a --047d7b10cf2d9b440c0528170c4a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable thanks=EF=BC=81 On Tue, Dec 29, 2015 at 12:14 PM, Edward Capriolo wrote: > hive --service lineage 'hql' exists i believe. > > On Tue, Dec 29, 2015 at 3:05 PM, Yang wrote: > >> I'm trying to create a utility to parse out the data lineage (i.e. DAG >> dependency graph) among all my hive scripts. >> >> to do this I need to parse out the input and output tables from a query. >> does this ability existing already? if not, I'm going to hack the parser= . >> I am not very familiar with the parser code structure of hive, could >> anybody give me some tips on where to start? >> (I see the .g files, but not sure where is the rest ---- I am more >> familiar with the ASTvisitor paradigm in antlr, but can't find similar >> files in the parser dir) >> >> >> thanks >> Yang >> > > --047d7b10cf2d9b440c0528170c4a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
thanks=EF=BC=81


On Tue, Dec 29, 2015 at 12:14 PM, Edwar= d Capriolo <edlinuxguru@gmail.com> wrote:
hive --service lineage 'hql' e= xists i believe.

On Tue, Dec 29, 2015 at 3:05 P= M, Yang <teddyyyy123@gmail.com> wrote:
I'm trying to create a utility to par= se out the data lineage (i.e. DAG dependency graph) among all my hive scrip= ts.

to do this I need to parse out the input and output = tables from a query.=C2=A0
does this ability existing already? if= not, I'm going to hack the parser.
I am not very familiar wi= th the parser code structure of hive, could anybody give me some tips on wh= ere to start?
(I see the .g files, but not sure where is the rest= ---- I am more familiar with the ASTvisitor paradigm in antlr, but can'= ;t find similar files in the parser dir)


thanks
Yang


--047d7b10cf2d9b440c0528170c4a--