hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: [DISCUSS] Remove append?
Date Fri, 23 Mar 2012 00:41:34 GMT
On Thu, Mar 22, 2012 at 4:27 PM,  <Milind.Bhandarkar@emc.com> wrote:
> Eli,
>
> I think by "current definition of visible length", you mean that once a
> client opens a file and gets block list, it will always be able to read up
> to the length at open.
>

I was thinking of the definition from the design doc. See my last
comment on HDFS-2288, part of the confusion is that we're using the
same name for two different things.

> However, correct me if I am wrong, but this definition is already
> violated, if file is deleted after open.

I think you're right.

> So, truncate does add some complexity, but not a whole lot. If client gets
> an EOF before length at open, it must retry to see if the new visible
> length is different (rather than to see if the file does not exist
> anymore).
>
> Right ?
>

Makes sense. I was thinking you were talking about truncate on open
files, which be harder.  You can already truncate a file on open, you
just can't choose the offset you want to truncate at (the NN
implements this by deleting the file).

Thanks,
Eli

>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
>
> On 3/22/12 4:03 PM, "Eli Collins" <eli@cloudera.com> wrote:
>
>>On Thu, Mar 22, 2012 at 3:57 PM, Tsz Wo Sze <szetszwo@yahoo.com> wrote:
>>>> Do you think having the invariant that blocks are not mutated would
>>>> significantly simply the design?
>>>
>>> No.  As mentioned in my previous email and others, the complexity is in
>>>hflush.  Once we have hflush, append is straightforward.
>>
>>I understand that append is a small delta once you have hflush, what
>>I'm saying is that the overall design of the file system is
>>significantly simplified if you can assume blocks are not mutated. Eg
>>see the way truncate is going to interact with the current definition
>>of visible length (it violates it). Resolving issues like that are
>>non-trivial.
>>
>>Thanks,
>>Eli
>>
>

Mime
View raw message