Return-Path: 
 <core-dev-return-71720-apmail-hadoop-core-dev-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org
Received: (qmail 76752 invoked from network); 18 Jun 2009 16:26:26 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 18 Jun 2009 16:26:26 -0000
Received: (qmail 12675 invoked by uid 500); 18 Jun 2009 16:26:34 -0000
Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org
Received: (qmail 12570 invoked by uid 500); 18 Jun 2009 16:26:34 -0000
Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:core-dev-help@hadoop.apache.org>
List-Unsubscribe: <mailto:core-dev-unsubscribe@hadoop.apache.org>
List-Post: <mailto:core-dev@hadoop.apache.org>
List-Id: <core-dev.hadoop.apache.org>
Reply-To: core-dev@hadoop.apache.org
Delivered-To: mailing list core-dev@hadoop.apache.org
Received: (qmail 12551 invoked by uid 99); 18 Jun 2009 16:26:34 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Jun 2009 16:26:34 +0000
X-ASF-Spam-Status: No, hits=2.2 required=10.0
	tests=HTML_MESSAGE,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of pareekash@gmail.com
 designates 209.85.221.178 as permitted sender)
Received: from [209.85.221.178] (HELO mail-qy0-f178.google.com)
 (209.85.221.178)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Jun 2009 16:26:24 +0000
Received: by qyk8 with SMTP id 8so1560156qyk.5
        for <multiple recipients>; Thu, 18 Jun 2009 09:26:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:received:in-reply-to:references
         :date:message-id:subject:from:to:content-type;
        bh=YXU8mWmwgdRvVZ/XgAsgXN/iwCMDFfwe6oQbhKQevUs=;
        b=MnjUoYMQFNmQ5A0skRAdEVHxSQ6KHcxv8Ush+tp7qvSCPHUPk3WlGv4d9UU+eOTDyH
         9VbQnvWEUxSrdXT+YHMH0VPHCf/bfcQNr4QfTbCz3GuKKI+RMEJCZGBKMaNZjL3qUesL
         uRKNVg+GwMpzejDJuUHShU7zTxHAJi5NUhZHc=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=bXxTI6wUYlUJOjxE7rcH+n18bpPPU2JMIM89RYN7u3YkydtfZaXvva7h6Rq+e+smhv
         tMeFnT5r7i8FvY8XByelzTRMpjfoSrzuDD5yTzXuqG1l+9mbwKZwFos7+d2gMgs3hIqu
         QWM+N1uQ45Yu3l1GDY14r0O/yAa7laeiMTGk0=
MIME-Version: 1.0
Received: by 10.224.32.153 with SMTP id c25mr1508633qad.184.1245342363401;
	Thu, 18 Jun 2009 09:26:03 -0700 (PDT)
In-Reply-To: <314098690906180906x26b32b02w8935a7ca1a5456c9@mail.gmail.com>
References: <45d9159d0906172202v4d0554f8h1b39f9a1408e021b@mail.gmail.com>
	 <45d9159d0906180746j5819db2ey607deebf226376a0@mail.gmail.com>
	 <314098690906180800t75716208l2c7d6146a95685ff@mail.gmail.com>
	 <45d9159d0906180810s24651773la54a1d365f5a081a@mail.gmail.com>
	 <314098690906180823q67a7733ev95bb797c09abc222@mail.gmail.com>
	 <45d9159d0906180833j3e427bf2r55c0d6dc9234f077@mail.gmail.com>
	 <314098690906180906x26b32b02w8935a7ca1a5456c9@mail.gmail.com>
Date: Thu, 18 Jun 2009 21:56:03 +0530
Message-ID: <45d9159d0906180926q4746fcccmb8c5b05f38ec873c@mail.gmail.com>
Subject: Re: Need help
From: ashish pareek <pareekash@gmail.com>
To: core-dev@hadoop.apache.org, core-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001517576edc31bef2046ca1ddd7
X-Virus-Checked: Checked by ClamAV on apache.org

--001517576edc31bef2046ca1ddd7
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hello Everybody,

                          How can we handle different applications having
different requirement being run on the same hadoop cluster ? What are the
various approaches to solve such problem.. if possible please mention some
of those ideas.

Does such implementation exists ?

Thanks ,

Regards,
Ashish

On Thu, Jun 18, 2009 at 9:36 PM, jason hadoop <jason.hadoop@gmail.com>wrote:

> For me, I like to have one configuration file that I distribute to all of
> the machines in my cluster via rsync.
>
> In there are things like the number of tasks per node to run, and where to
> store dfs data and local temporary data, and the limits to storage for the
> machines.
>
> If the machines are very different, it becomes important to tailor the
> configuration file per machine or type of machine.
>
> At this point, you are pretty much going to have to spend the time, reading
> through the details of configuring a hadoop cluster.
>
>
> On Thu, Jun 18, 2009 at 8:33 AM, ashish pareek <pareekash@gmail.com>
> wrote:
>
> > Can you tell few of the challenges in configuring heterogeneous
> > cluster...or
> > can pass on some link where I would get some information regarding
> > challenges in running Hadoop on heterogeneous hardware
> >
> > One more things is How about running different applications on the same
> > Hadoop cluster?and what challenges are involved in it ?
> >
> > Thanks,
> > Regards,
> > Ashish
> >
> >
> > On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop <jason.hadoop@gmail.com
> > >wrote:
> >
> > > I don't know anyone who has a completely homogeneous cluster.
> > >
> > > So hadoop is scalable across heterogeneous environments.
> > >
> > > I stated that configuration is simpler if the machines are similar
> (There
> > > are optimizations in configuration for near homogeneous machines.)
> > >
> > > On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pareekash@gmail.com>
> > > wrote:
> > >
> > > > Does that mean hadoop is not scalable wrt heterogeneous environment?
> > and
> > > > one
> > > > more question is can we run different application on the same hadoop
> > > > cluster
> > > > .
> > > >
> > > > Thanks.
> > > > Regards,
> > > > Ashish
> > > >
> > > > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <
> jason.hadoop@gmail.com
> > > > >wrote:
> > > >
> > > > > Hadoop has always been reasonably agnostic wrt hardware and
> > > homogeneity.
> > > > > There are optimizations in configuration for near homogeneous
> > machines.
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <
> pareekash@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >        I am doing my master.... my final year project is on
> Hadoop
> > > > ...so
> > > > > I
> > > > > > would like to know some thing about Hadoop cluster i.e, Do new
> > > version
> > > > of
> > > > > > Hadoop are able to handle heterogeneous hardware.If you have any
> > > > > > informantion regarding these please mail me as my project is in
> > > > > > heterogenous
> > > > > > environment.
> > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Reagrds,
> > > > > > Ashish Pareek
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > > > www.prohadoopbook.com a community for Hadoop Professionals
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > www.prohadoopbook.com a community for Hadoop Professionals
> > >
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>

--001517576edc31bef2046ca1ddd7--