hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: Flushing to HDFS sooner
Date Wed, 22 Feb 2012 01:29:15 GMT
I tried on trunk, and this scenario seems to works fine.

In fact I first forgot to enable appends (I just switched my local end to hadoop-1.0.0), and
without that I did in fact loose the edits.
With appends enables this works as designed.

Might still be a timing issue or only occur in 0.90.x.

Maybe somebody else could mine the attached logs for clues?

-- Lars

 From: lars hofhansl <lhofhansl@yahoo.com>
To: Manuel de Ferran <manuel.deferran@gmail.com>; "user@hbase.apache.org" <user@hbase.apache.org>

Sent: Tuesday, February 21, 2012 4:38 PM
Subject: Re: Flushing to HDFS sooner
You still should not loose data this way.
Looks like something that is easily reproducible. I'll try with the latest trunk.

-- Lars

From: Manuel de Ferran <manuel.deferran@gmail.com>
To: user@hbase.apache.org 
Cc: lars hofhansl <lhofhansl@yahoo.com> 
Sent: Tuesday, February 21, 2012 3:51 AM
Subject: Re: Flushing to HDFS sooner

On Mon, Feb 20, 2012 at 9:43 PM, Stack <stack@duboce.net> wrote:

On Mon, Feb 20, 2012 at 11:58 AM, lars hofhansl <lhofhansl@yahoo.com> wrote:
>> Are there any messages about log replay when you restart the region server?
>What Lars says', whats it say in the logs on master on restart?

I just noticed that if I restart the master before starting the regionserver, I still have
my rows and I have "recovered.edits" logs in the master log :

2012-02-21 10:09:55,541 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://lxc167.nightly-dev.com:9100/hbase/.META./1028785192/recovered.edits/0000000000000029079 (wrote
4 edits in 22ms)

During my previous tests, I was doing the following :
- kill datanode
- start datanode
- start regionserver
- restart master (because .META. and -ROOT- were the only regions online): i did not realize
that I could loose any data doing that.

Here are the master logs to both cases :
http://pastebin.com/KD03P0pD : master restart before regionserver
http://pastebin.com/FvYaBMdm : master restart after regionserver
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message