Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
From: Venkatesh S <svenkat@yahoo-inc.com>
To: "general@hadoop.apache.org" <general@hadoop.apache.org>
Date: Fri, 10 Dec 2010 10:06:29 +0530
Subject: Re: Multiple hadoop configurations
Thread-Topic: Multiple hadoop configurations
Thread-Index: AcuYCBpsaHt3cv0IQwOPsFTDgRClxAAG7WDM
Message-ID: <C927AC25.4939C%svenkat@yahoo-inc.com>
In-Reply-To: <AANLkTim0M9+Y-Yc9GR=9AtTZEGVTSuPrFMHTvNp6Kpn5@mail.gmail.com>
Accept-Language: en-US
Content-Language: en
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_C927AC254939Csvenkatyahooinccom_"
MIME-Version: 1.0

--_000_C927AC254939Csvenkatyahooinccom_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

If you are not running the same version, you could use Reverse Class Loader=
 trick to be able to launch jobs to multiple clusters.

-Venkatesh


On 12/10/10 6:46 AM, "Alejandro Abdelnur" <tucu@cloudera.com> wrote:

Deepika,

If both of your clusters run the same version of Hadoop, then -as
Konstantin suggested- you could use Oozie. Oozie does not rely on
local Hadoop configuration files to determine the JT/NN to use, you
specify them in the Oozie workflow application XML for each job.

Cheers.

Alejandro

On Fri, Dec 10, 2010 at 7:01 AM, C J <c.joshi26@yahoo.com> wrote:
> Thanks Konstantin. My issue is more with supporting the multiple clusters=
 . I am
> using quartz for scheduling the jobs (currently it is 2 separate schedule=
rs for
> the 2 clusters).
>
>
> If while submitting a job I am able to specify which cluster it should tr=
igger
> the job on (by giving the handle to the appropriate cluster), I think I w=
ill be
> able to manage it.
>
> Thanks,
> Deepika
>
>
>
>
> ________________________________
> From: Konstantin Boudnik <cos@apache.org>
> To: general@hadoop.apache.org
> Sent: Thu, December 9, 2010 2:24:34 PM
> Subject: Re: Multiple hadoop configurations
>
> I believe the answer you are looking for is Oozie coordinator, but I
> am not which version of it supports multiple clusters.
>
> On Thu, Dec 9, 2010 at 14:20, C J <c.joshi26@yahoo.com> wrote:
>> Hi,
>>
>> I have 2 hadoop clusters . Both these clusters have their own set of job=
s
>> running. I also have some distcp jobs which copy over data from one clus=
ter to
>> another.
>>
>> I want to be able to control the jobs on both the clusters through one
>>scheduler
>> (so I can coordinate the jobs).
>>
>> I am wondering when I need to trigger a job through a scheduler, how can=
 I
> send
>> to one of the 2 clusters.
>>
>>
>> 2 clusters means 2 sets of configuration files.Is there any way to get a
> handle
>> to one of these clusters, by specifying the configuration file name or
>> something?
>>
>> Will appreciate any help or clues.
>>
>> Thanks.
>>
>>
>>
>>
>
>
>
>


--_000_C927AC254939Csvenkatyahooinccom_--