Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=O5XULEkGenzbeCBGCGDhHKtEHtl5rqA9Rr8aKGz0zdrNaH2rz6VOQpQe5rGglEnjXay6H0d6ph1u4FKKh27m6t2g3tpCnQmaTaKIk37Ij4Brbu84PpebPE3W3tnZoYkpQigsoV5M3EoTfcHp3IGbgoJ1W86L2CQjR94xeBzWnVA=;
References: 
 <CAHau4yt_Q_Q2t9N=513VgBC4GrsoCFpdTh9Tc8hRA4o7Buofbw@mail.gmail.com>
 <CAGWW7L=gNV+VLNJ2nzJ0ppt7bD_zGOLi7XNVkqwhbWb_sKuxbw@mail.gmail.com>
 <CAHau4ysF0gYUnZGwXTYunqyHSKZKYKS47ZDjvzXbJv_6o=uMig@mail.gmail.com>
 <2D6136772A13B84E95DF6DA79E85A9F0016B66255696@NSPEXMBX-A.the-lab.llnl.gov>
Message-ID: <1317244420.85268.YahooMailNeo@web121705.mail.ne1.yahoo.com>
Date: Wed, 28 Sep 2011 14:13:40 -0700 (PDT)
From: lars hofhansl <lhofhansl@yahoo.com>
Reply-To: lars hofhansl <lhofhansl@yahoo.com>
Subject: Re: Recommended backup/restore solution for hbase
To: "user@hbase.apache.org" <user@hbase.apache.org>
In-Reply-To: 
 <2D6136772A13B84E95DF6DA79E85A9F0016B66255696@NSPEXMBX-A.the-lab.llnl.gov>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="-1714046436-1972296751-1317244420=:85268"

---1714046436-1972296751-1317244420=:85268
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

You can now (0.92+) set the minium number of versions you want to always ke=
ep around together with TTL. See HBASE-4071 =0A=0A=0A-- Lars=0A=0A=0A=0A___=
_____________________________=0AFrom: "Buttler, David" <buttler1@llnl.gov>=
=0ATo: "user@hbase.apache.org" <user@hbase.apache.org>=0ASent: Wednesday, S=
eptember 28, 2011 2:10 PM=0ASubject: RE: Recommended backup/restore solutio=
n for hbase=0A=0AWouldn't using a TTL on your data automatically delete dat=
a that is older than X months?=A0 Of course major compactions have to occur=
 to get the data to automatically disappear.=0A=0ASee:=0Ahttp://hbase.apach=
e.org/book.html#ttl=0Ahttp://hbase.apache.org/apidocs/org/apache/hadoop/hba=
se/HColumnDescriptor.html#HColumnDescriptor(byte[], int, java.lang.String, =
boolean, boolean, int, int, java.lang.String, int)=0A=0ADave=0A=0A-----Orig=
inal Message-----=0AFrom: tvinod@socialyantra.com [mailto:tvinod@socialyant=
ra.com] On Behalf Of Vinod Gupta Tankala=0ASent: Wednesday, September 28, 2=
011 12:12 PM=0ATo: user@hbase.apache.org=0ASubject: Re: Recommended backup/=
restore solution for hbase=0A=0Athanks Li. I didn't know about using S3 as =
a datastore. Will look into this=0Amore.=0A=0AI understand that hdfs replic=
ation will help in partial hardware failure. I=0Awanted to protect myself a=
gainst inconsistencies as I have gotten bitten in=0Athe past. That had happ=
ened due to hbase fatal exceptions. One of the=0Areasons for that could hav=
e been due to standalone mode as that is not=0Aproduction ready, based on r=
eading hbase documentation.=0AAnother use case I have is - I would be writi=
ng sweeper jobs to delete user=0Adata that is more than x months old. So in=
 case, we need to retrieve old=0Auser data, I would like to have the abilit=
y to get old data back from=0Aexported tables. Ofcourse, I understand that =
to do so for selective user=0Aaccounts, I have to write custom jobs.=0A=0At=
hanks=0Avinod=0A=0AOn Wed, Sep 28, 2011 at 11:49 AM, Li Pi <lpi@ucsd.edu> w=
rote:=0A=0A> What kind of situations are you looking for to guard against? =
Partial=0A> hardware failure, full hardware failure (of live cluster),=0A> =
accidentally deleting all data?=0A>=0A> HDFS provides replication that alre=
ady guards against partial hardware=0A> failure - if this is all you need, =
a ephemeral store should be=A0 fine.=0A>=0A> Also, HBase can use S3 directl=
y as a datastore. You can choose the raw=0A> mode, in which HBase treats S3=
 as a disk. There used to be a block=0A> based mode as well, but now as S3 =
has increased the object size limit=0A> to 5tb, this isn't needed anymore. =
(Somebody correct me if i'm wrong).=0A>=0A> On Wed, Sep 28, 2011 at 9:15 AM=
, Vinod Gupta Tankala=0A> <tvinod@readypulse.com> wrote:=0A> > Hi,=0A> > Ca=
n someone answer these basic but important questions for me.=0A> > We are u=
sing hbase for our datastore and want to safeguard ourselves from=0A> > dat=
a corruption/data loss. Also we are hosted on aws ec2. Currently, I=0A> onl=
y=0A> > have a single node but want to prepare for scale right away as thin=
gs are=0A> > going to change starting next couple of weeks. Also, I am curr=
ently using=0A> > ephemeral store for hbase data.=0A> >=0A> > 1) What is th=
e recommended aws data store method for hbase? should you=0A> use=0A> > eph=
emeral store and do S3 backups or use EBS? I read and heard that EBS=0A> ca=
n=0A> > be expensive and also unreliable in terms of read/write latency.=0A=
> Ofcourse,=0A> > it provides data replication and protection, so you don't=
 have to worry=0A> > about that.=0A> >=0A> > 2) What is the recommended bac=
kup/restore method for hbase? I would like=0A> to=0A> > take periodic data =
snapshots and then have a import utility that will=0A> > incrementally impo=
rt data in case i lose some regions due to corruption=0A> or=0A> > table in=
consistencies. also, if something catastrophic happens, i can=0A> > restore=
 the whole data.=0A> >=0A> > 3) While we are at it, what is the recommended=
 ec2 instance types for=0A> > running master/zookeeper/region servers? i ge=
t conflicting answers from=0A> > google search - ranging from c1.xlarge to =
m1.xlarge.=0A> >=0A> > I would really appreciate if someone could help me.=
=0A> >=0A> > thanks=0A> > vinod=0A> >=0A>
---1714046436-1972296751-1317244420=:85268--