Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of mohajeri@gmail.com designates
 209.85.214.170 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAMbwnT8YbLinh2G-CjFoJbBxJE6GEmSxsxGszbcBGhpON09jXg@mail.gmail.com>
References: 
 <CAMbwnT8YbLinh2G-CjFoJbBxJE6GEmSxsxGszbcBGhpON09jXg@mail.gmail.com>
Date: Tue, 17 Sep 2013 14:43:23 -0700
Message-ID: 
 <CAO6JcpjoSGT9yztcFBgMbkJMMyVPTBBQADcFWG13R7VeiXVzaA@mail.gmail.com>
Subject: Re: Oozie dynamic action
From: Peyman Mohajerian <mohajeri@gmail.com>
To: user@hadoop.apache.org
Cc: "oozie-users@incubator.apache.org" <oozie-users@incubator.apache.org>
Content-Type: multipart/alternative; boundary=089e0149c506c2264e04e69b35bf

--089e0149c506c2264e04e69b35bf
Content-Type: text/plain; charset=ISO-8859-1

If you want to see a simple example of what you are looking for:
https://github.com/cloudera/cdh-twitter-example
It is part of this article:
http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-with-hadoop/


On Tue, Sep 17, 2013 at 4:20 AM, praveenesh kumar <praveenesh@gmail.com>wrote:

> Hi,
>
> I have a scenario in which I want to trigger a hive uploading script every
> day. I have a set of folders created for a set of customer ids everyday. My
> hive script will read the customer id from the path, checks whether the
> table for the customer id exits and if not create a table and will create
> partition based on date for  a "set of unknown customer_ids".
>
> I can get the set of unique customer_ids from a shell action. It can be
> passed as a list or string.
>
> My problem is how can I achieve this dynamic checking/creation of hive
> tables and partitions from oozie.
>
> Currently I am doing everything from the shell script and calling it as a
> shell action in oozie, but I was wondering if these kind of checks or some
> kind of for-loop kind of actions can be done in oozie.
>
> Any thoughts/suggestions on how to tackle the above scenario in the best
> way possible using oozie, would be highly helpful.
>
> Regards
> Praveenesh
>
>

--089e0149c506c2264e04e69b35bf
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>If you want to see a simple example of what you are l=
ooking for:<br><a href=3D"https://github.com/cloudera/cdh-twitter-example">=
https://github.com/cloudera/cdh-twitter-example</a><br></div>It is part of =
this article:<br>
<a href=3D"http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-wit=
h-hadoop/">http://blog.cloudera.com/blog/2012/09/analyzing-twitter-data-wit=
h-hadoop/</a><br></div><div class=3D"gmail_extra"><br><br><div class=3D"gma=
il_quote">
On Tue, Sep 17, 2013 at 4:20 AM, praveenesh kumar <span dir=3D"ltr">&lt;<a =
href=3D"mailto:praveenesh@gmail.com" target=3D"_blank">praveenesh@gmail.com=
</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin=
:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir=3D"ltr"><div><div><div><div><div><div><div><div>Hi,<br><br></div>I=
 have a scenario in which I want to trigger a hive uploading script every d=
ay. I have a set of folders created for a set of customer ids everyday. My =
hive script will read the customer id from the path, checks whether the tab=
le for the customer id exits and if not create a table and will create part=
ition based on date for=A0 a &quot;set of unknown customer_ids&quot;.<br>

</div><br></div>I can get the set of unique customer_ids from a shell actio=
n. It can be passed as a list or string.<br><br></div>My problem is how can=
 I achieve this dynamic checking/creation of hive tables and partitions fro=
m oozie.<br>

<br></div>Currently I am doing everything from the shell script and calling=
 it as a shell action in oozie, but I was wondering if these kind of checks=
 or some kind of for-loop kind of actions can be done in oozie.<br><br>

</div>Any thoughts/suggestions on how to tackle the above scenario in the b=
est way possible using oozie, would be highly helpful.<br><br></div>Regards=
<span class=3D"HOEnZb"><font color=3D"#888888"><br></font></span></div><spa=
n class=3D"HOEnZb"><font color=3D"#888888">Praveenesh<br>
<div><div><div><div><div><div><br></div></div></div>
</div></div></div></font></span></div>
</blockquote></div><br></div>

--089e0149c506c2264e04e69b35bf--