hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Laurent Chavet (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3804) HBase gets stuck with "Regions in transition timed out" for .META. and -ROOT-
Date Thu, 21 Apr 2011 17:44:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022842#comment-13022842
] 

Laurent Chavet commented on HBASE-3804:
---------------------------------------

Thanks very much; that was it.
I am running in "pseudo-distributed" and the region server did die (I need to increase the
limit on open files).
It all makes sense to me now; thanks for the very fast answers.
BTW: is there standard way to automatically restart the region severs when they crash or should
I write my own small script to deal with that.
Actually do you even think it's a good idea to automatically restart (I might have missed
the real problem)?

> HBase gets stuck with "Regions in transition timed out" for .META. and -ROOT-
> -----------------------------------------------------------------------------
>
>                 Key: HBASE-3804
>                 URL: https://issues.apache.org/jira/browse/HBASE-3804
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.90.2
>         Environment: CentOs 5.4 x64 kernel 2.6.18-238.5.1.el5
> Single node cluster
>            Reporter: Laurent Chavet
>            Priority: Blocker
>
> After putting some significant load on the sever (load around of 8 for approximatively
50 minutes) the following starts showing in the logs and then hbase is completely stuck: won't
recover without a restart (I copy a large chunk of the master logs before the problem in case
it helps; the bottom part about ROOT and META repeats forever):
> 9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305484717
> 2011-04-20 06:19:51,228 INFO org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter:
Using syncFs -- HDFS-200
> 2011-04-20 06:19:51,228 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter:
Creating writer path=hdfs://hqsnosq
> l01:9000/hbase/Property/69e1f895ca9d6824b409015a9f9c03a3/recovered.edits/0000000000000098012
region=69e1f895ca9d6824b409
> 015a9f9c03a3
> 2011-04-20 06:19:51,539 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter:
Pushed=38 entries from hdfs://HQSNO
> SQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305484717
> 2011-04-20 06:19:51,540 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter:
Splitting hlog 42 of 43: hdfs://HQS
> NOSQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251,
length=67522024
> 2011-04-20 06:19:51,540 INFO org.apache.hadoop.hbase.util.FSUtils: Recovering file hdfs://HQSNOSQL01:9000/hbase/.logs/HQ
> SNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251
> 2011-04-20 06:19:52,543 INFO org.apache.hadoop.hbase.util.FSUtils: Finished lease recover
attempt for hdfs://HQSNOSQL01:
> 9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251
> 2011-04-20 06:19:53,336 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter:
Pushed=40 entries from hdfs://HQSNO
> SQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251
> 2011-04-20 06:19:54,894 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter:
Splitting hlog 43 of 43: hdfs://HQS
> NOSQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805,
length=75765855
> 2011-04-20 06:19:54,906 INFO org.apache.hadoop.hbase.util.FSUtils: Recovering file hdfs://HQSNOSQL01:9000/hbase/.logs/HQ
> SNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805
> 2011-04-20 06:19:55,108 INFO org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter:
Using syncFs -- HDFS-200
> 2011-04-20 06:19:55,108 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter:
Creating writer path=hdfs://hqsnosq
> l01:9000/hbase/Property/a21bf162e9ff2915ff036f486919c9b5/recovered.edits/0000000000000098062
region=a21bf162e9ff2915ff03
> 6f486919c9b5
> 2011-04-20 06:19:55,908 INFO org.apache.hadoop.hbase.util.FSUtils: Finished lease recover
attempt for hdfs://HQSNOSQL01:
> 9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805
> 2011-04-20 06:19:56,842 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter:
Pushed=43 entries from hdfs://HQSNO
> SQL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805
> 2011-04-20 06:19:56,859 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305143661
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305143661
> 2011-04-20 06:19:56,863 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305152287
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305152287
> 2011-04-20 06:19:56,867 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305160979
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305160979
> 2011-04-20 06:19:56,871 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305170430
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305170430
> 2011-04-20 06:19:56,875 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305178662
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305178662
> 2011-04-20 06:19:56,879 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305188551
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305188551
> 2011-04-20 06:19:56,883 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305196703
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305196703
> 2011-04-20 06:19:56,887 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305205039
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305205039
> 2011-04-20 06:19:56,891 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305213412
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305213412
> 2011-04-20 06:19:56,895 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305220665
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305220665
> 2011-04-20 06:19:56,899 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305231697
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305231697
> 2011-04-20 06:19:56,913 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305239174
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305239174
> 2011-04-20 06:19:56,917 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305248013
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305248013
> 2011-04-20 06:19:56,921 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305256411
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305256411
> 2011-04-20 06:19:56,925 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305265663
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305265663
> 2011-04-20 06:19:56,929 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305274667
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305274667
> 2011-04-20 06:19:56,933 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305283140
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305283140
> 2011-04-20 06:19:56,937 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305291307
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305291307
> 2011-04-20 06:19:56,941 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305300632
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305300632
> 2011-04-20 06:19:56,945 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305309439
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305309439
> 2011-04-20 06:19:56,949 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305316405
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305316405
> 2011-04-20 06:19:56,953 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305322730
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305322730
> 2011-04-20 06:19:56,957 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305330453
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305330453
> 2011-04-20 06:19:56,961 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305337020
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305337020
> 2011-04-20 06:19:56,965 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305345309
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305345309
> 2011-04-20 06:19:56,969 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305352766
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305352766
> 2011-04-20 06:19:56,972 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305362335
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305362335
> 2011-04-20 06:19:56,976 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305370474
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305370474
> 2011-04-20 06:19:56,980 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305379360
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305379360
> 2011-04-20 06:19:56,984 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305387434
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305387434
> 2011-04-20 06:19:56,988 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305396557
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305396557
> 2011-04-20 06:19:56,992 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305403848
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305403848
> 2011-04-20 06:19:56,996 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305412950
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305412950
> 2011-04-20 06:19:57,000 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305420349
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305420349
> 2011-04-20 06:19:57,004 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305429358
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305429358
> 2011-04-20 06:19:57,008 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305437383
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305437383
> 2011-04-20 06:19:57,012 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305446715
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305446715
> 2011-04-20 06:19:57,016 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305460169
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305460169
> 2011-04-20 06:19:57,678 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305467377
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305467377
> 2011-04-20 06:19:57,685 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305476873
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305476873
> 2011-04-20 06:19:57,689 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305484717
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305484717
> 2011-04-20 06:19:57,693 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305492251
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305492251
> 2011-04-20 06:19:57,697 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Archived
processed log hdfs://HQSNOS
> QL01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146/HQSNOSQL01%3A60020.1303305500805
to hdfs://hqsnosql01:9000/hbase/.o
> ldlogs/HQSNOSQL01%3A60020.1303305500805
> 2011-04-20 06:19:57,701 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Waiting
for split writer threads to
> finish
> 2011-04-20 06:19:57,874 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Split
writers finished
> 2011-04-20 06:19:57,881 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/36d89fc46915d8db56bad45c64a313a3/recovered.edits/0000000000000096979 (wrote
134 edits in 6107ms)
> 2011-04-20 06:19:57,885 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/4499b405bae1047884256f39b7559376/recovered.edits/0000000000000096571 (wrote
30 edits in 1338ms)
> 2011-04-20 06:19:57,889 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/465fb6b0a1f2d5306c7bb87145a32be2/recovered.edits/0000000000000096568 (wrote
22 edits in 909ms)
> 2011-04-20 06:19:57,893 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/4bc3c0bd3a824fb6c7fabbd332ef1009/recovered.edits/0000000000000097151 (wrote
85 edits in 3982ms)
> 2011-04-20 06:19:57,897 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/4bfff6ceb59cc4b6098eea42b49ee9ed/recovered.edits/0000000000000097336 (wrote
117 edits in 4810ms)
> 2011-04-20 06:19:57,901 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/52dee69cbe0890b54aee56bda54577c9/recovered.edits/0000000000000096567 (wrote
63 edits in 4445ms)
> 2011-04-20 06:19:57,905 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/539e4afe8064897395d4dc891605e702/recovered.edits/0000000000000097127 (wrote
53 edits in 3433ms)
> 2011-04-20 06:19:57,909 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/5728cb0bb67c7c3d8701fc0679db97b9/recovered.edits/0000000000000097473 (wrote
134 edits in 6623ms)
> 2011-04-20 06:19:57,913 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/5f92d0c1defc108140df27322257584d/recovered.edits/0000000000000096672 (wrote
77 edits in 7211ms)
> 2011-04-20 06:19:57,917 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/69e1f895ca9d6824b409015a9f9c03a3/recovered.edits/0000000000000098012 (wrote
10 edits in 306ms)
> 2011-04-20 06:19:57,921 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/70746a106ab152f290264d0b2b0f773c/recovered.edits/0000000000000096722 (wrote
117 edits in 7245ms)
> 2011-04-20 06:19:57,925 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/906e3283419ae67052c34e31cc40ff05/recovered.edits/0000000000000097951 (wrote
31 edits in 2389ms)
> 2011-04-20 06:19:57,929 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/94112f821b90271f9d693d5505d5eebf/recovered.edits/0000000000000097692 (wrote
71 edits in 4771ms)
> 2011-04-20 06:19:57,933 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/9f519dfd7370990d017e7f6d187968d3/recovered.edits/0000000000000096697 (wrote
77 edits in 5551ms)
> 2011-04-20 06:19:57,937 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/9f60e52d384d0fd9dfc2a2f3be42ac43/recovered.edits/0000000000000097877 (wrote
24 edits in 840ms)
> 2011-04-20 06:19:58,054 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/a21bf162e9ff2915ff036f486919c9b5/recovered.edits/0000000000000098062 (wrote
8 edits in 1860ms)
> 2011-04-20 06:19:58,061 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/abc26969d71b22de80c7e889cc9a6c12/recovered.edits/0000000000000096576 (wrote
5 edits in 281ms)
> 2011-04-20 06:19:58,065 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/b5d7c2bd8383ea8871890f40b9827dcf/recovered.edits/0000000000000097554 (wrote
125 edits in 5240ms)
> 2011-04-20 06:19:58,073 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/bf06b369cf84e5bccdd4e845636a2af6/recovered.edits/0000000000000096839 (wrote
1 edits in 17ms)
> 2011-04-20 06:19:58,077 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/cf46d330cdaa1120c8e1a5fcf1899b0f/recovered.edits/0000000000000097442 (wrote
1 edits in 850ms)
> 2011-04-20 06:19:58,081 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/d850f84bfdfa28c63934a0be5a41257e/recovered.edits/0000000000000096572 (wrote
53 edits in 2467ms)
> 2011-04-20 06:19:58,085 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/d9b94307130a7096032b44b3aa18d3ea/recovered.edits/0000000000000096795 (wrote
155 edits in 10534ms)
> 2011-04-20 06:19:58,088 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/da6540fdaae565a6d04cd78a145240dd/recovered.edits/0000000000000097580 (wrote
53 edits in 2992ms)
> 2011-04-20 06:19:58,092 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/df02c5d923c466c5b89337fecee920cf/recovered.edits/0000000000000096924 (wrote
1 edits in 25ms)
> 2011-04-20 06:19:58,096 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Closed
path hdfs://hqsnosql01:9000/h
> base/Property/e90c1c020e3148756a57d977133e31fc/recovered.edits/0000000000000096569 (wrote
96 edits in 3862ms)
> 2011-04-20 06:19:58,097 INFO org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: hlog
file splitting completed in 866
> 02 ms for hdfs://hqsnosql01:9000/hbase/.logs/HQSNOSQL01,60020,1303302276146
> 2011-04-20 06:19:58,102 INFO org.apache.hadoop.hbase.catalog.RootLocationEditor: Unsetting
ROOT region location in ZooKe
> eper
> 2011-04-20 06:19:58,113 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12f72de7aee0000
Creating (or up
> dating) unassigned node for 70236052 with OFFLINE state
> 2011-04-20 06:19:58,133 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12f72de7aee0000
Creating (or up
> dating) unassigned node for 1028785192 with OFFLINE state
> 2011-04-20 06:19:58,139 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=M_ZK_REGION_OFFLINE,
>  server=HQSNOSQL01:60000, region=1028785192/.META.
> 2011-04-20 06:20:36,363 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305598133
> 2011-04-20 06:20:36,364 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:20:36,364 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305598113
> 2011-04-20 06:20:36,364 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:20:36,364 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305598133
> 2011-04-20 06:20:36,364 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305598113
> 2011-04-20 06:21:06,374 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305636364
> 2011-04-20 06:21:06,374 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:21:06,374 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305636364
> 2011-04-20 06:21:06,374 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:21:06,374 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305636364
> 2011-04-20 06:21:06,374 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305636364
> 2011-04-20 06:21:36,385 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305666374
> 2011-04-20 06:21:36,385 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:21:36,385 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  -ROOT-, 
> ,0.70236052 state=OFFLINE, ts=1303305666374
> 2011-04-20 06:21:36,385 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:21:36,385 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305666374
> 2011-04-20 06:21:36,385 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305666374
> 2011-04-20 06:22:06,393 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305696385
> 2011-04-20 06:22:06,393 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:22:06,393 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305696385
> 2011-04-20 06:22:06,393 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:22:06,393 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305696385
> 2011-04-20 06:22:06,393 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305696385
> 2011-04-20 06:22:36,402 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305726393
> 2011-04-20 06:22:36,402 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:22:36,402 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305726394
> 2011-04-20 06:22:36,402 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:22:36,402 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305726393
> 2011-04-20 06:22:36,402 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305726394
> 2011-04-20 06:23:06,411 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305756402
> 2011-04-20 06:23:06,411 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:23:06,411 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305756402
> 2011-04-20 06:23:06,411 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:23:06,412 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305756402
> 2011-04-20 06:23:06,412 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305756402
> 2011-04-20 06:23:36,420 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305786412
> 2011-04-20 06:23:36,420 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:23:36,420 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305786412
> 2011-04-20 06:23:36,420 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:23:36,420 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=.META.,,1.102878519
> 2 state=OFFLINE, ts=1303305786412
> 2011-04-20 06:23:36,420 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=-ROOT-,,0.70236052
> state=OFFLINE, ts=1303305786412
> 2011-04-20 06:24:06,429 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  .META.,
> ,1.1028785192 state=OFFLINE, ts=1303305816420
> 2011-04-20 06:24:06,430 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning .META.,,1.1028785192 to a random server
> 2011-04-20 06:24:06,430 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions
in transition timed out:  -ROOT-,
> ,0.70236052 state=OFFLINE, ts=1303305816420
> 2011-04-20 06:24:06,430 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region
has been OFFLINE for too long, rea
> ssigning -ROOT-,,0.70236052 to a random server
> 2011-04-20 06:24:06,430 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing
OFFLINE; was=.META.,,1.102878519

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message