Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1EB4317DE8 for ; Mon, 4 May 2015 07:37:24 +0000 (UTC) Received: (qmail 30680 invoked by uid 500); 4 May 2015 07:37:18 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 30544 invoked by uid 500); 4 May 2015 07:37:18 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 30534 invoked by uid 99); 4 May 2015 07:37:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 May 2015 07:37:18 +0000 X-ASF-Spam-Status: No, hits=2.4 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: message received from 54.164.171.186 which is an MX secondary for user@hadoop.apache.org) Received: from [54.164.171.186] (HELO mx1-us-east.apache.org) (54.164.171.186) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 May 2015 07:37:11 +0000 Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id E1F9B43E4C for ; Mon, 4 May 2015 07:36:50 +0000 (UTC) Received: by lbbuc2 with SMTP id uc2so99007313lbb.2 for ; Mon, 04 May 2015 00:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=xYlFBgjvB03HSkJB2Jpgs/GTVy88iVXP/zZuA/L4gVI=; b=mCLvFV8xhdABFMkWFbcOHErP+zP5D6oIhqiZ0IbziCoyzNlrVQ618dVt3o3T+I/KkZ EJKn+mv4owH6QcUbbTnGpvFJm2OyolGAL6d/PW5NUpGK/20P0Es0iJk/mVcPGpK4lwXm y68f9dx/ANEZ95kx90MAD9Y3KoqiGPImQO7c3LfzrKvcU1aP19pMRDj/qWJOxKfq0B+A 4HZUl44JXMvphfWEgU+xqpgSmbuK7T366jjKWu6U1qBKMPC/R+4RDd6iaaZ8XMh8ggnF wOLZiX5MQTlZu5hOMyYAoIS8PjqA9A5ZdYTyJ65I4ax25giRHwq9H9D9qURk4/W4zqQ0 4JIQ== MIME-Version: 1.0 X-Received: by 10.112.29.180 with SMTP id l20mr17799055lbh.95.1430724919706; Mon, 04 May 2015 00:35:19 -0700 (PDT) Received: by 10.152.123.164 with HTTP; Mon, 4 May 2015 00:35:19 -0700 (PDT) In-Reply-To: References: Date: Mon, 4 May 2015 13:05:19 +0530 Message-ID: Subject: Re: Can we control data distribution and load balancing in Hadoop Cluster? From: Answer Agrawal To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a113402e09c46ff05153c9acc X-Virus-Checked: Checked by ClamAV on apache.org --001a113402e09c46ff05153c9acc Content-Type: text/plain; charset=UTF-8 Thanks Mr Chandrashekhar The input data sets in HDFS breaks it in blocks of default size 128 MB and replicate it by default replication factor 3. It also balance load by transfering job of failed or busy nodes to free or active nodes. Can we manage how much data sets and load should assign to which node by ourselves. On Mon, May 4, 2015 at 12:03 AM, Chandrashekhar Kotekar < shekhar.kotekar@gmail.com> wrote: > Your question is very vague. Can you give us more details about the > problem you are trying to solve? > > > Regards, > Chandrash3khar Kotekar > Mobile - +91 8600011455 > > On Sun, May 3, 2015 at 11:59 PM, Answer Agrawal > wrote: > >> Hi >> >> As I studied that data distribution, load balancing, fault tolerance are >> implicit in Hadoop. But I need to customize it, can we do that? >> >> Thanks >> >> > --001a113402e09c46ff05153c9acc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Mr Chandrashekhar

The in= put data sets in HDFS breaks it in blocks of default size 128 MB and replic= ate it by default replication factor 3. It also balance load by transfering= job of failed or busy nodes to free or active nodes. Can we manage how muc= h data sets and load should assign to which node by ourselves.

On Mon, May 4= , 2015 at 12:03 AM, Chandrashekhar Kotekar <shekhar.kotekar@gmail.= com> wrote:
Your question is very vague. Can you give us more details about the probl= em you are trying to solve?


Regards,
Chandrash3khar Kotekar
Mobile - +91 8600011455

On Sun, May 3, 2015 at 11:59 PM, Answer Agra= wal <yrsna.tset01@gmail.com> wrote:
Hi=C2=A0

As I s= tudied that data distribution, load balancing, fault tolerance are implicit= in Hadoop. But I need to customize it, can we do that?

Thanks=C2=A0



--001a113402e09c46ff05153c9acc--