hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arbow <avin...@gmail.com>
Subject Re: How is big file got divided
Date Thu, 20 Apr 2006 06:56:16 GMT
Hi, Lei Chen:

  You can have a view on org.apache.hadoop.mapred.InputFormatBase, I
think it will help you.

On 4/20/06, Lei Chen <lchen5@gmail.com> wrote:
> Hi,
>      I am a new user of hadoop. This project looks cool.
>      There is one question about the MapReduce. I want to process a big
> file. To my understanding, hadoop will partition big file into block and
> each block is assigned to a worker. Then, how does hadoop decide where to
> cut those big files? Does it guarantee that each line in the input file will
> be assigned to one block and no line will be divided into two parts in
> different blocks?
> Lei

View raw message