Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of
 bharathvissapragada1990@gmail.com designates 209.85.210.194 as permitted
 sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        b=XRMLC+CMr+x3NTd+gmbQ/qr2JIc0FrXGEL8m1DRTzN3kYmhpgGjKzhXu2VQ8ok97Aa
         bP76gURLT/2HIcgGznUoF1AcmHqkbr/ul8FnzUcgRRxSu11iDzEUg0ecS4pXM8AHdpHa
         557w/JjdM1Qe+j/Ma1bdfolnEbuxsag65p32c=
MIME-Version: 1.0
Sender: bharathvissapragada1990@gmail.com
In-Reply-To: <88FC9367-5F65-46FA-B17D-BEB87667D26E@wensel.net>
References: <73d592f60910020329h1baa1c36w3554b850b950ab9b@mail.gmail.com>
	 <88FC9367-5F65-46FA-B17D-BEB87667D26E@wensel.net>
Date: Sat, 3 Oct 2009 22:59:00 +0530
Message-ID: <73d592f60910031029g5f286809me451e6d641b370b7@mail.gmail.com>
Subject: Re: Cascading jobs in hadoop
From: bharath vissapragada <bharat_v@students.iiit.ac.in>
To: common-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Tom and Chris ,

Thanks for your replies .. I have seen thr o.a.h.mapred.jobcontrol.Job
and o.a.h.mapreduce.Job .. Only one of them has the above option of
adding a dependent Jobs .. Can anyone tell me the difference between
"mapred" and "mapreduce" packages ..

Thanks in advance

On 10/2/09, Chris K Wensel <chris@wensel.net> wrote:
> You might find the Cascading project quite useful in this regard.
>
> http://www.cascading.org/
>
> using MapReduceFlow and CascadeConnector classes, you can chain
> arbitrary MR jobs together. Cascading will determine the dependencies,
> if any, and run the jobs in topological order (independent jobs will
> be submitted to run in parallel).
>
> you may also find writing your own MR jobs by hand tedious and
> brittle. Cascading can help you there as well.
>
> cheers,
> chris
>
> On Oct 2, 2009, at 3:29 AM, bharath v wrote:
>
>> Hi all,
>>
>> I have a set of map red jobs which need to be cascaded ,i.e, output
>> of MR
>> job1 is the input of MR job2. etc..
>>
>> Can anyone point me to the corresponding classes in hadoop 0.20.0 API?
>>
>> I have seen "x.addDependingJob(y)" function in the yahoo's hadoop
>> tutorial
>> but that is for the older versions..
>> What is the similar thing in 0.20.0 API?
>>
>> Any help is appreciated ,
>>
>> Thanks
>> bharath.v
>> ug3
>> IIIT Hyderabad!
>
> --
> Chris K Wensel
> chris@concurrentinc.com
> http://www.concurrentinc.com
>
>