hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bai, Gang" <...@baigang.net>
Subject Re: A way to monitor HDFS for a file to come live, and then kick off a job?
Date Fri, 25 Mar 2011 14:25:40 GMT
Hi Jon,

Oozie could handle this nicely. You may just specify a Oozie coordinator
jobs. But if you don't have a Oozie server handy, cron jobs could also meet
your needs.

Regards,
-BaiGang

On Fri, Mar 25, 2011 at 1:09 AM, Jonathan Coveney <jcoveney@gmail.com>wrote:

> I am not sure if this is the right listserv, forgive me if it is not. My
> goal is this: monitor HDFS until a file is create, and then kick off a job.
> Ideally I'd want to do this continuously, but the file would be create
> hourly (with some sort of variance). I guess I could make a script that
> would ping the server every 5 minutes or something, but I was wondering if
> there might be a more elegant way?
>
> Thanks
> Jon
>

Mime
View raw message