Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2EF3B11624 for ; Tue, 20 May 2014 09:13:59 +0000 (UTC) Received: (qmail 42841 invoked by uid 500); 20 May 2014 09:13:57 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 42783 invoked by uid 500); 20 May 2014 09:13:57 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 42775 invoked by uid 99); 20 May 2014 09:13:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 May 2014 09:13:57 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of santhosh.thomas@yahoo.com designates 98.139.212.155 as permitted sender) Received: from [98.139.212.155] (HELO nm25-vm1.bullet.mail.bf1.yahoo.com) (98.139.212.155) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 May 2014 09:13:51 +0000 Received: from [98.139.212.152] by nm25.bullet.mail.bf1.yahoo.com with NNFMP; 20 May 2014 09:13:30 -0000 Received: from [98.139.212.240] by tm9.bullet.mail.bf1.yahoo.com with NNFMP; 20 May 2014 09:13:30 -0000 Received: from [127.0.0.1] by omp1049.mail.bf1.yahoo.com with NNFMP; 20 May 2014 09:13:30 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 129725.20017.bm@omp1049.mail.bf1.yahoo.com Received: (qmail 54073 invoked by uid 60001); 20 May 2014 09:13:30 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1400577210; bh=MG94jnBJCO+s8oTrPZZq78xBalbKwHoxx6iAbScqkkw=; h=References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=lCQPvz6/qTt84thbAJoxBQdTGgS18/Je6oRmX27O4fSjzwFkLk/3ZQV/59Le5wDP7WWPZ8OLle16gkZ4wOkLFAzSaNq2n1MW94dUDGDVJ11TkA0Td6ZxtybjWZB4BKuqAAGGlyhJ/b6xIkdxYrm1dchXUoRTQjkt+b3MkrKZMDY= X-YMail-OSG: EPjHxkMVM1l44sSNWTOiAVtkqJQ9evhOQjjAT0l1Rg9SbBX nhdClzZHTpQYpmLnjx1vG7U5y3aBewTb_rJq47HoRZlbJOIDeZ3GM7jmhowL 4YzBw9M2067cPHy9XsFuVFLMugzRaILY3pg0SGM2V8BfONDtE58PKuo50NHY LYw8kxYaXc.R6y6.rm.6cTOo891Ys2Lpj2HF7Hz2pFLoRUVSBsDnssbnwCqS IXSIAgOhwRrwu5lQjrk8FuYJuy5UJSW6jw5OBYXUkB1ug43ORZY_s48iEWEq 5VHhfPNtnPy2yJaDOPQeXF60JIn311VEvb4mjmiu3a4W7wlA_Y1LQ1L6UxS0 GLMwGCCfbF7PEy.LfkEyHqtM5JoV7DU75LBi.2HwbbFbHkZ8qVtX2_T8BV3A bTZe_5w.SUzH58gU1zz_VoKy5u8nEpSq3Uf36gnQpGcoySoaqX6UOMl.ZQoZ kxESplICK5l_eq.6qBkO8zVQTYa0kH6hhYPKecAhPVAIaEp8qY5nVgzkbPqw JYOMlt5bHtPwf0w_6pkU_GBLtJF7izqm5XwCXv6K9Y2btFKNyzA-- Received: from [196.15.16.101] by web141402.mail.bf1.yahoo.com via HTTP; Tue, 20 May 2014 02:13:29 PDT X-Rocket-MIMEInfo: 002.001,VGhhbmtzIFN1bmlsLiBJIGFtIGFscmVhZHkgY2xvc2UgdG_CoCBpdC4gV2hlcmUgZGlkIHlvdSBwdXQgdGhlIE5vdGlmaWNhdGlvbkxpc3RlbmVyLmNsYXNzPwrCoAp.c2FudGhvc2gKIAogRnJvbTogU3VuaWwgTiBLdW1hciA8c3VuaWxuLmt1bWFyQGltcGV0dXMuY28uaW4.ClRvOiAidXNlckBoaXZlLmFwYWNoZS5vcmciIDx1c2VyQGhpdmUuYXBhY2hlLm9yZz47IFNhbnRob3NoIFRob21hcyA8c2FudGhvc2gudGhvbWFzQHlhaG9vLmNvbT4gCkNjOiBSYXB0b3JzIDxTdGFtZm9yZC1QbGF0Zm9ybUBpbXBldHUBMAEBAQE- X-Mailer: YahooMailWebService/0.8.188.663 References: <1400571980.65247.YahooMailNeo@web141405.mail.bf1.yahoo.com> <2eec3260f0ec49deb8fc289e2df80dfa@MBX3.impetus.co.in> Message-ID: <1400577209.78702.YahooMailNeo@web141402.mail.bf1.yahoo.com> Date: Tue, 20 May 2014 02:13:29 -0700 (PDT) From: Santhosh Thomas Reply-To: Santhosh Thomas Subject: Re: How to detect new table creation in hive To: Sunil N Kumar , "user@hive.apache.org" Cc: Raptors In-Reply-To: <2eec3260f0ec49deb8fc289e2df80dfa@MBX3.impetus.co.in> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-409376090-1221977848-1400577209=:78702" X-Virus-Checked: Checked by ClamAV on apache.org ---409376090-1221977848-1400577209=:78702 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Thanks Sunil. I am already close to=A0 it. Where did you put the Notificati= onListener.class?=0A=A0=0A~santhosh=0A =0A From: Sunil N Kumar =0ATo: "user@hive.apache.org" ; Santh= osh Thomas =0ACc: Raptors =0ASent: Tuesday, May 20, 2014 2:22 PM=0ASubject: RE: How to d= etect new table creation in hive=0A =0A=0A=0AHi Santhosh, =0AI have used t= his in the Falcon for creating Hive process to manage Hive tables across mu= ltiple Hadoop cluster. Current Hcatalog publish an event to the Message Bro= ker say Active MQ whenever new partition added or deleted in the Hive Metas= tore. Following setting needs to done at hive-site.xml. This is not documen= ted in the Hive documnetion. But I have used this and it is working as expe= cted. It create topic in the activeMQ as =0AHcat..= . =0A =0Ahive.metastore.event.listeners =0Aor= g.apache.hcatalog.listener.NotificationListener =0A =0A =0A=A0 =0Ahttps://cwiki.apache.org/confluence/di= splay/Hive/HCatalog+Notification =0AHCatalog provides notifications for cer= tain events happening in the system. This way applications such as Oozie ca= n wait for those events and schedule the work that depends on them. The cur= rent version of HCatalog supports two kinds of events: =0A=B7=A0=A0=A0=A0= =A0=A0=A0=A0 Notification when a new partition is added =0A=B7=A0=A0=A0=A0= =A0=A0=A0=A0 Notification when a set of partitions is added =0A=A0 =0A=A0 = =0AFrom:Santhosh Thomas [mailto:santhosh.thomas@yahoo.com] =0ASent: Tuesday= , May 20, 2014 1:16 PM=0ATo: user@hive.apache.org=0ASubject: How to detect = new table creation in hive =0A=A0 =0A=A0 =0AHello =0A=A0 =0AI am writi= ng an application that detects new table creation in=A0 hive. =0AI came ac= ross attribute ' hive.metastore.event.listeners' in hive-default.xml, but c= ould not find any usage samples. =0AThe API docs also does not describe m= uch. Can someone please help? Is there a better way to detect new table cre= ation in hive? =0A=A0 =0Athanks =0ASanthosh =0A =0A=0A=0A=0A=0A=0ANO= TE: This message may contain information that is confidential, proprietary,= privileged or otherwise protected by law. The message is intended solely f= or the named addressee. If received in error, please destroy and notify the= sender. Any use of this email=0A is prohibited when received in error. Imp= etus does not represent, warrant and/or guarantee, that the integrity of th= is communication has been maintained nor that the communication is free of = errors, virus, interception or interference. ---409376090-1221977848-1400577209=:78702 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks Sunil. I am already close to  it. Wher= e did you put the NotificationListener.class?
 
~santhosh

From: Suni= l N Kumar <suniln.kumar@impetus.co.in>
To: "user@hive.apache.org" <user@hive.apache.org>;= Santhosh Thomas <santhosh.thomas@yahoo.com>
Cc: Raptors <Stamford-Platform@impetus.co.i= n>
Sent: Tuesday, = May 20, 2014 2:22 PM
Subject: RE: How to detect new table creation in hive

=0A
=0A
Hi Santhosh,
=0A
=0A= I have used this in the Falcon for creating Hive process to manage Hive tab= les across multiple Hadoop cluster. Current Hcatalog publish an event to th= e Message Broker say Active=0A MQ whenever new partition added or deleted i= n the Hive Metastore. Following setting needs to done at hive-site.xml. Thi= s is not documented in the Hive documnetion. But I have used this and it is= working as expected. It create topic in the activeMQ as
=0A=0AHcat.<Databasename>.<tablename>.<= /span>
=0A
=0A<property>=0A
=0A<name>hive.metastore.event.= listeners</name>
=0A
=0A&l= t;value>org.apache.hcatalog.listener.NotificationListener</value><= /span>
=0A
=0A<description></d= escription>
=0A
=0A</prope= rty>
=0A
=0A =0A=0A
=0AHCat= alog provides notifications for certain events happening in the system. Thi= s way applications such as Oozie can wait for those events and schedule the= work that depends on=0A them. The current version of HCatalog supports two= kinds of events:
=0A
=0A=B7    =     =0ANotification when a new partition is added
=0A
=0A= =B7        =0A<= /span>Notification when a set of partition= s is added
=0A
 
=0A
 
=0A
=0A
=0A
From: Santhos= h Thomas [mailto:santhosh.thomas@yahoo.com]=0A
=0ASent= : Tuesday, May 20, 2014 1:16 PM
=0ATo: user@hi= ve.apache.org
=0ASubject: How to detect new table = creation in hive
=0A
=0A
=0A
 
=0A
= =0A
=0A
 
=0A=0A
=0A
=0A
Hello=
=0A
=0A
=0A
 
=0A
=0A
=0A
I am = writing an application that detects new table creation in  hive.
=0A
=0A
=0A
I came a= cross attribute ' hive.metastore.event.listeners' in hive-default.xml, but = could not find any usage samples.=0A
=0A
=0A
=0A
The API docs also does not describe much.= Can someone please help? Is there a better way to detect new table creatio= n in hive?
=0A
=0A
=0A
thanks
=0A
=0A
=0A
Santhosh
=0A
=0A
=0A
=0A=0A
=0A
=0A
=0A
=0A
=0A
=0A
=0A
=0ANOTE: This message may contain information that is confide= ntial, proprietary, privileged or otherwise protected by law. The message i= s intended solely for the named addressee. If received in error, please des= troy and notify the sender. Any use of this email=0A is prohibited when rec= eived in error. Impetus does not represent, warrant and/or guarantee, that = the integrity of this communication has been maintained nor that the commun= ication is free of errors, virus, interception or interference.
=0A=0A


= ---409376090-1221977848-1400577209=:78702--