hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santhosh Thomas <santhosh.tho...@yahoo.com>
Subject Re: How to detect new table creation in hive
Date Tue, 20 May 2014 09:13:29 GMT
Thanks Sunil. I am already close to  it. Where did you put the NotificationListener.class?
 
~santhosh
 
 From: Sunil N Kumar <suniln.kumar@impetus.co.in>
To: "user@hive.apache.org" <user@hive.apache.org>; Santhosh Thomas <santhosh.thomas@yahoo.com>

Cc: Raptors <Stamford-Platform@impetus.co.in> 
Sent: Tuesday, May 20, 2014 2:22 PM
Subject: RE: How to detect new table creation in hive
  


Hi Santhosh, 
I have used this in the Falcon for creating Hive process to manage Hive tables across multiple
Hadoop cluster. Current Hcatalog publish an event to the Message Broker say Active MQ whenever
new partition added or deleted in the Hive Metastore. Following setting needs to done at hive-site.xml.
This is not documented in the Hive documnetion. But I have used this and it is working as
expected. It create topic in the activeMQ as 
Hcat.<Databasename>.<tablename>. 
<property> 
<name>hive.metastore.event.listeners</name> 
<value>org.apache.hcatalog.listener.NotificationListener</value> 
<description></description> 
</property> 
  
https://cwiki.apache.org/confluence/display/Hive/HCatalog+Notification 
HCatalog provides notifications for certain events happening in the system. This way applications
such as Oozie can wait for those events and schedule the work that depends on them. The current
version of HCatalog supports two kinds of events: 
·         Notification when a new partition is added 
·         Notification when a set of partitions is added 
  
  
From:Santhosh Thomas [mailto:santhosh.thomas@yahoo.com] 
Sent: Tuesday, May 20, 2014 1:16 PM
To: user@hive.apache.org
Subject: How to detect new table creation in hive   
  
   
Hello  
   
I am writing an application that detects new table creation in  hive.  
I came across attribute ' hive.metastore.event.listeners' in hive-default.xml, but could not
find any usage samples.   
The API docs also does not describe much. Can someone please help? Is there a better way to
detect new table creation in hive?  
   
thanks  
Santhosh     
 





NOTE: This message may contain information that is confidential, proprietary, privileged or
otherwise protected by law. The message is intended solely for the named addressee. If received
in error, please destroy and notify the sender. Any use of this email
 is prohibited when received in error. Impetus does not represent, warrant and/or guarantee,
that the integrity of this communication has been maintained nor that the communication is
free of errors, virus, interception or interference.
Mime
View raw message