hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phantom <ghostwhoowa...@gmail.com>
Subject Re: hdfsOpenFile() API
Date Wed, 13 Jun 2007 23:07:11 GMT
Which would mean that if I want to have my logs to reside in HDFS I will
have to move them using copyFromLocal or some version thereof and then run
Map/Reduce process against them ? Am I right ?

Thanks
Avinash

On 6/13/07, Owen O'Malley <oom@yahoo-inc.com> wrote:
>
>
> On Jun 13, 2007, at 3:29 PM, Phantom wrote:
>
> > Hmm I was under the impression that HDFS is like GFS optimized for
> > appends
> > although GFS supports random writes.
>
> HDFS doesn't support appends. There has been discussion of
> implementing single-writer appends, but it hasn't reached the top of
> anyone's priority list. Some people (me included) aren't thrilled by
> the semantics of atomic append in GFS. To me, it seems like atomic
> append is basically a poor-man's map/reduce. *smile*
>
> -- Owen
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message