Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <BANLkTim=fCa-R1ibTunHVhcjjgExukQBZA@mail.gmail.com>
References: <BANLkTim=fCa-R1ibTunHVhcjjgExukQBZA@mail.gmail.com>
From: Ted Dunning <tdunning@maprtech.com>
Date: Thu, 14 Apr 2011 16:23:48 -0700
Message-ID: <BANLkTi=97c7ao2EC5Qq+6U2GLCLikw9fFg@mail.gmail.com>
Subject: Re: silent data loss during append
To: hdfs-user@hadoop.apache.org
Cc: Thanh Do <thanhdo@cs.wisc.edu>, hdfs-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0023547c8ae5d3b6bc04a0e938a0

--0023547c8ae5d3b6bc04a0e938a0
Content-Type: text/plain; charset=ISO-8859-1

What version are you using?

On Thu, Apr 14, 2011 at 3:55 PM, Thanh Do <thanhdo@cs.wisc.edu> wrote:

> Hi all,
>
> I have recently seen silent data loss in our system.
> Here is the case:
>
> 1. client appends to some block
> 2. for some reason, commitBlockSynchronization
>     returns successfully with synclist = [] (i.e empty)
> 3. in the client code, NO exception is thrown, and
>    client appends successfully.
> 4. However, the block replicas are then removed from
>    datanodes, causing data loss.
>
> Have any one seen this before?
> Is this behavior by design or a bug?
>
> Many thanks,
> Thanh
>
>
>
>

--0023547c8ae5d3b6bc04a0e938a0--