hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hairong Kuang <kuang.hair...@gmail.com>
Subject Re: Reading while appending in 0.21
Date Tue, 12 Jan 2010 01:42:24 GMT
This semantics is stated in our append specification attached to HDFS-265.
Hflushed data are visible to new readers who open the file after the hflush.

Hairong

On Mon, Jan 11, 2010 at 5:33 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Thanks for the answer Konstantin, good to know. I also see that the
> commenter doesn't seem pleased with the fact that the file has to be
> reopened ;)
>
> So basically what I wrongly expected was a tailing feature. Searches
> in Jira don't give me a hit, would it make sense to open one?
>
> In the mean time, I'm ok with recreating the reader.
>
> Thx,
>
> J-D
>
> On Mon, Jan 11, 2010 at 5:15 PM, Konstantin Boudnik <cos@yahoo-inc.com>
> wrote:
> > Jean,
> >
> > I believe this is how it has been intended to work. If you take a look at
> > the test src/test/hdfs/org/apache/hadoop/hdfs/TestHFlush.java in HDFS
> > workspace you'll a comment on this particular aspect in the line 124.
> >
> > Hope it helps,
> >  Cos
> >
> > On 1/11/10 16:42 , Jean-Daniel Cryans wrote:
> >>
> >> Hi,
> >>
> >> I'm trying to use the new hflush function from 0.21 so that a
> >> SequenceFile.Reader could read edits from a SequenceFile.Writer after
> >> a signal on a Condition. If I:
> >>
> >> create the Writer
> >> append entries
> >> hflush
> >> create the Reader
> >> next() through the entries
> >>
> >> It works fine. But after that if I only next(), using the same reader,
> >> after appending/hflush I won't see the new edits. But, if I create a
> >> new Reader after calling hflush, it works fine.
> >>
> >> So this does not work:
> >>
> >> create the Writer
> >> append entries
> >> hflush
> >> create the Reader
> >> next() through the entries
> >> append entries
> >> hflush
> >> next() through the entries
> >> append entries
> >> hflush
> >> next() through the entries
> >>
> >> This does:
> >>
> >> create the Writer
> >> append entries
> >> hflush
> >> create the Reader
> >> next() through the entries
> >> append entries
> >> hflush
> >> create a new Reader
> >> seek
> >> next() through the entries
> >>
> >> Is that the intended behavior?
> >>
> >> Thx!
> >>
> >> J-D
> >
>

Mime
View raw message