Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of eli@cloudera.com designates
 209.85.214.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <CB90FF9D.2E109%Milind.Bhandarkar@emc.com>
References: 
 <CAPn_vTvK8pU+_bneo-fBkt6hGcTAa5AZMvSJyY29NUn4-_fMXA@mail.gmail.com>
	<CB90FF9D.2E109%Milind.Bhandarkar@emc.com>
Date: Thu, 22 Mar 2012 17:41:34 -0700
Message-ID: 
 <CAPn_vTtgyFRXuPMofrb=A9Fg5PtJ-UrVtrGHH2J7qrut2Xsp3w@mail.gmail.com>
Subject: Re: [DISCUSS] Remove append?
From: Eli Collins <eli@cloudera.com>
To: hdfs-dev@hadoop.apache.org
Cc: szetszwo@yahoo.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Thu, Mar 22, 2012 at 4:27 PM,  <Milind.Bhandarkar@emc.com> wrote:
> Eli,
>
> I think by "current definition of visible length", you mean that once a
> client opens a file and gets block list, it will always be able to read u=
p
> to the length at open.
>

I was thinking of the definition from the design doc. See my last
comment on HDFS-2288, part of the confusion is that we're using the
same name for two different things.

> However, correct me if I am wrong, but this definition is already
> violated, if file is deleted after open.

I think you're right.

> So, truncate does add some complexity, but not a whole lot. If client get=
s
> an EOF before length at open, it must retry to see if the new visible
> length is different (rather than to see if the file does not exist
> anymore).
>
> Right ?
>

Makes sense. I was thinking you were talking about truncate on open
files, which be harder.  You can already truncate a file on open, you
just can't choose the offset you want to truncate at (the NN
implements this by deleting the file).

Thanks,
Eli

>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, an=
d
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
>
> On 3/22/12 4:03 PM, "Eli Collins" <eli@cloudera.com> wrote:
>
>>On Thu, Mar 22, 2012 at 3:57 PM, Tsz Wo Sze <szetszwo@yahoo.com> wrote:
>>>> Do you think having the invariant that blocks are not mutated would
>>>> significantly simply the design?
>>>
>>> No. =A0As mentioned in my previous email and others, the complexity is =
in
>>>hflush. =A0Once we have hflush, append is straightforward.
>>
>>I understand that append is a small delta once you have hflush, what
>>I'm saying is that the overall design of the file system is
>>significantly simplified if you can assume blocks are not mutated. Eg
>>see the way truncate is going to interact with the current definition
>>of visible length (it violates it). Resolving issues like that are
>>non-trivial.
>>
>>Thanks,
>>Eli
>>
>