hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prabhu Hari Dhanapal <dragonzsn...@gmail.com>
Subject Re: File split query
Date Fri, 29 Jan 2010 06:55:09 GMT
The splitting does not know anything about the input file's internal logical
structure, for example line-oriented text files are split on arbitrary byte
boundaries.

On Fri, Jan 29, 2010 at 1:49 AM, .ke. sivakumar <kesivakumar@gmail.com>wrote:

> Hadoop will take care of it. If the split is supposed to be at the middle
> of
> the
> line, then it will be extended till the end. Though the split limit will be
> exceeded
> by few bytes.
>
>
>
> On Thu, Jan 28, 2010 at 7:34 PM, Udaya Lakshmi <udaya603@gmail.com> wrote:
>
> > Hi,
> >   When framework splits a file, will it happen that some part of a
> > line falls in one split and the other part in some other split? Or is
> > the framework going to take care that it always splits at the end of
> > the line?
> >
> > Thanks,
> > Udaya.
> >
>



-- 
Hari

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message