Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of saint.ack@gmail.com
 designates 209.85.216.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:cc:content-type
         :content-transfer-encoding;
        b=uzWLKWXlvtgVcUvdpKTXQ4MHr/YOgmn918O+6Ls+dzEcCXcuypY7o4Ru1O/J/c7n2C
         gLAbD5mlxuD9piR7OG6IqAV40lObFNAnuXYHlhW++IB5WNMpDS/wki/zioWoSyK62MAL
         s2VX5EhU+Feaj+j/Ftzc8FgLTMkjqGBxInZgk=
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: <4DB9AB17.30408@opendns.com>
References: <4DB9AB17.30408@opendns.com>
Date: Thu, 28 Apr 2011 16:28:28 -0700
Message-ID: <BANLkTi=8whCv4S=V_McO1YAHCFSuYTRLfQ@mail.gmail.com>
Subject: Re: LoadIncrementalHFiles now deleting the hfiles?
From: Stack <stack@duboce.net>
To: user@hbase.apache.org
Cc: CDH Users <cdh-user@cloudera.org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I took a look through the code and don't see any explicit removes and
looking through history of changes to the file, I don't see any change
of substance.

Can you figure what is doing the delete? At what stage?  Is it as
completebulkload runs?

St.Ack

On Thu, Apr 28, 2011 at 10:59 AM, Adam Phelps <amp@opendns.com> wrote:
> We were using a backup scheme for our system where we have map-reduce job=
s
> generating HFiles, which we then loaded using LoadIncrementalHFiles befor=
e
> making a remote copy of them using distcp.
>
> However we just upgraded hbase (we're using cloudera's package, so we wen=
t
> from CDH3B4 to CDH3U0, both of which are versions of 0.90.1), and discove=
red
> that the HFiles now get deleted by the load operation. =A0Is this a recen=
t
> change? =A0Is there a configuration variable to revert this behavior?
>
> We can work around it by doing the copy before the load, but that is less
> than optimal in our scenario as we'd prefer to have quicker access to the
> data in HBase.
>
> - Adam
>