hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <>
Subject RE: Hive CLI and Standalone Server : Need Suggestion
Date Mon, 19 Mar 2012 19:38:50 GMT
Great topic as I was wondering a similar thing this morning...I want to use oozie to execute
my hive job, but I have to pass the job parameters that I generate with a shell script.  Some
of the literature that I've seen says that oozie may or may not allow for calling shell scripts.
 Is that true?


From: ext Bejoy Ks []
Sent: Monday, March 19, 2012 15:34
Subject: Re: Hive CLI and Standalone Server : Need Suggestion

Hi LakshmiKanth
        In production systems if you have a sequence of command to be executed pack them in
order in a file. Then execute the command as
hive -f <filename> ;

For simplicity, you can use a cron job to run it in a scheduled manner. Just give this command
in a .sh file call the file in cron. Infact you can use any scheduler that would trigger a
.sh file.

But for hadoop based work flows the preferred workflow manager is oozie and I recommend oozie
for hadoop jobs.

Bejoy KS

From: LakshmiKanth P <<>>
Sent: Tuesday, March 20, 2012 12:19 AM
Subject: Hive CLI and Standalone Server : Need Suggestion


I need to schedule my hive scripts which needs to process incoming weblogs on an hourly basis.

Currently, I could process my weblog files by executing my scripts from hive command line
interface.  Now I want to keep my scripts in a file and invoke my scripts at a regular periods
of interval.  I came to know that hive command line options provides a facility to pass the
.sql file as input for execution.  Is it the right approach for any production environment.


Should I use my hive server in stand alone mode and inovke my hive scripts using JDBC API.

Request you to suggest me the best approach.


View raw message