hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject [ANN] HiveServiceBAction different take on Hive->Oozie integration
Date Sat, 28 Apr 2012 18:01:04 GMT
The current integration between oozie and hive runs a fat client on a
task tracker to launch the action. This is an issue in our environment
as we typically do not allow every task tracker to connect to our
mysql metastore. I created a Java main action that submits task to
hive thrift server. Enjoy:

https://github.com/edwardcapriolo/m6d_oozie

It works like this:
   <action name="create-node">
        <java>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <main-class>com.m6d.oozie.HiveServiceBAction</main-class>
            <arg>rhiveservice.hadoop.pvt</arg>
            <arg>10000</arg>
            <arg>CREATE TABLE IF NOT EXISTS zz_zz_abc (a int, b int)</arg>
            <arg>QUERY 2</arg>
            <arg>QUERY N</arg>
        </java>
        <ok to="query_node"/>
        <error to="fail"/>
    </action>

Mime
View raw message