Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of manoj444@gmail.com designates
 209.85.214.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <C76F0EA9-052E-4277-8A20-74985C8E7DC8@hortonworks.com>
References: 
 <CACvfG=qjton_-fWYv40ZnW-LWao8dk=LuL+n+dxyXhtd+MSKQg@mail.gmail.com>
 <C76F0EA9-052E-4277-8A20-74985C8E7DC8@hortonworks.com>
From: Manoj Babu <manoj444@gmail.com>
Date: Wed, 11 Jul 2012 18:17:41 +0530
Message-ID: 
 <CACvfG=qBYQ3KQVcKmvM-MdjYzZaZj=RTEn=PNyKeEDKDGGgEow@mail.gmail.com>
Subject: Re: Mapper basic question
To: mapreduce-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=000e0ce03e0ade1d3104c48d41d4

--000e0ce03e0ade1d3104c48d41d4
Content-Type: text/plain; charset=ISO-8859-1

Hi  Tariq \Arun,

The no of blocks(splits) = *total no of file size/hdfs block size *
replicate value*
The no of splits is again nothing but the blocks here.

Other than increasing the block size(input splits) is it possible to limit
that no of mappers?


Cheers!
Manoj.


On Wed, Jul 11, 2012 at 6:06 PM, Arun C Murthy <acm@hortonworks.com> wrote:

> Take a look at CombineFileInputFormat - this will create 'meta splits'
> which include multiple small spilts, thus reducing #maps which are run.
>
> Arun
>
> On Jul 11, 2012, at 5:29 AM, Manoj Babu wrote:
>
> Hi,
>
> The no of mappers is depends on the no of blocks. Is it possible to limit
> the no of mappers size without increasing the HDFS block size?
>
> Thanks in advance.
>
> Cheers!
> Manoj.
>
>
>  --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>

--000e0ce03e0ade1d3104c48d41d4
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi=A0
<span style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:1=
3px;background-color:rgb(255,255,255)">Tariq</span>=A0\Arun,<div><br></div>=
<div>The no of blocks(splits) =3D <b>total no of file size/hdfs block size =
* replicate value</b></div>

<div>The no of splits is again nothing but the blocks here.</div><div><br><=
/div><div>Other than increasing the block size(input splits) is it possible=
 to limit that no of mappers?</div><div><br></div><div><br clear=3D"all">

Cheers!<div>Manoj.</div><br>
<br><br><div class=3D"gmail_quote">On Wed, Jul 11, 2012 at 6:06 PM, Arun C =
Murthy <span dir=3D"ltr">&lt;<a href=3D"mailto:acm@hortonworks.com" target=
=3D"_blank">acm@hortonworks.com</a>&gt;</span> wrote:<br><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex">

<div style=3D"word-wrap:break-word">Take a look at CombineFileInputFormat -=
 this will create &#39;meta splits&#39; which include multiple small spilts=
, thus reducing #maps which are run.<div><br></div><div>Arun</div><div><div=
>

<div class=3D"h5"><br><div><div>On Jul 11, 2012, at 5:29 AM, Manoj Babu wro=
te:</div><br><blockquote type=3D"cite">Hi,<div><br></div><div>The no of map=
pers is depends on the no of blocks. Is it possible to limit the no of mapp=
ers size without increasing the HDFS block size?</div>

<div><br></div><div>Thanks in advance.</div><div><br clear=3D"all">

Cheers!<div>Manoj.</div><br>
</div>
</blockquote></div><br></div></div><div>
<span style=3D"text-indent:0px;letter-spacing:normal;font-variant:normal;te=
xt-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:norm=
al;border-collapse:separate;text-transform:none;font-size:medium;white-spac=
e:normal;font-family:Helvetica;word-spacing:0px"><span style=3D"text-indent=
:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font=
-style:normal;font-weight:normal;line-height:normal;border-collapse:separat=
e;text-transform:none;font-size:medium;white-space:normal;font-family:Helve=
tica;word-spacing:0px"><div style=3D"word-wrap:break-word">

<span style=3D"text-indent:0px;letter-spacing:normal;font-variant:normal;te=
xt-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:norm=
al;border-collapse:separate;text-transform:none;font-size:medium;white-spac=
e:normal;font-family:Helvetica;word-spacing:0px"><div style=3D"word-wrap:br=
eak-word">

--</div><div style=3D"word-wrap:break-word">Arun C. Murthy</div><div style=
=3D"word-wrap:break-word">Hortonworks Inc.<br><a href=3D"http://hortonworks=
.com/" target=3D"_blank">http://hortonworks.com/</a><br><br></div></span></=
div>
</span></span>
</div>
<br></div></div></blockquote></div><br></div>

--000e0ce03e0ade1d3104c48d41d4--