hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1573) Holes in master state change; updated startcode and server go into .META. but catalog scanner just got old values
Date Wed, 29 Jul 2009 01:03:14 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-1573:
-------------------------

    Fix Version/s: 0.20.0

Bringing into 0.20.0.

Rapleaf just got it on TRUNK.

I saw it last night at pset on a 0.19 install:

{code}
[stack@aa0-000-8 hbase]$ grep 'nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259' hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-*|grep
-v PROCESS_OPEN
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:49:56,674 INFO
org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_SPLIT: nyt,0yGU-ywdW8iPTKOVcFFxuV==,1248214827405:
nyt,0yGU-ywdW8iPTKOVcFFxuV==,1248214827405 split; daughters: nyt,0yGU-ywdW8iPTKOVcFFxuV==,1248216593259,
nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259: safeMode=false from XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:49:56,674 INFO
org.apache.hadoop.hbase.master.RegionManager: Assigning region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
to XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:49:59,698 INFO
org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259:
safeMode=false from XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:49:59,701 INFO
org.apache.hadoop.hbase.master.ProcessRegionOpen: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
open on XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:49:59,701 INFO
org.apache.hadoop.hbase.master.ProcessRegionOpen: updating row nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
in region .META.,,1 with  with startcode 1248174024124 and server XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:50:00,625 DEBUG
org.apache.hadoop.hbase.master.BaseScanner: Current assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
is not valid;  Server '' unknown.
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:50:02,723 INFO
org.apache.hadoop.hbase.master.RegionManager: Assigning region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
to XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:50:05,756 INFO
org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259:
safeMode=false from XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:50:05,757 INFO
org.apache.hadoop.hbase.master.ProcessRegionOpen: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
open on XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:50:05,757 INFO
org.apache.hadoop.hbase.master.ProcessRegionOpen: updating row nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
in region .META.,,1 with  with startcode 1248174024124 and server XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21 22:51:02,053 DEBUG
org.apache.hadoop.hbase.master.BaseScanner: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 no
longer has references to nyt,0yGU-ywdW8iPTKOVcFFxuV==,1248214827405
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 16:54:20,359 DEBUG
org.apache.hadoop.hbase.master.BaseScanner: Current assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
is not valid;  Server 'XX.XX.45.124:20020' unknown.
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 17:06:23,114 INFO
org.apache.hadoop.hbase.master.RegionManager: Assigning region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
to XX.XX.44.95:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 17:36:00,303 DEBUG
org.apache.hadoop.hbase.master.BaseScanner: Current assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
is not valid;  serverInfo: address: XX.XX.45.124:20020, startcode: 1248283082287, load: (requests=0,
regions=24, usedHeap=95, maxHeap=1820), passed startCode: 1248174024124, storedInfo.startCode:
1248283082287
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 17:36:01,033 INFO
org.apache.hadoop.hbase.master.RegionManager: Assigning region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
to XX.XX.44.133:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 17:36:37,427 INFO
org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259:
safeMode=false from XX.XX.44.133:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 19:53:24,884 DEBUG
org.apache.hadoop.hbase.master.BaseScanner: Current assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
is not valid;  serverInfo: address: XX.XX.45.124:20020, startcode: 1248286712292, load: (requests=0,
regions=28, usedHeap=113, maxHeap=1820), passed startCode: 1248174024124, storedInfo.startCode:
1248286712292
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 19:53:24,886 INFO
org.apache.hadoop.hbase.master.RegionManager: Assigning region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
to XX.XX.44.185:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 19:53:28,708 INFO
org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259:
safeMode=false from XX.XX.44.185:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 19:58:36,599 INFO
org.apache.hadoop.hbase.master.ProcessRegionOpen: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
open on XX.XX.44.185:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 19:58:36,599 INFO
org.apache.hadoop.hbase.master.ProcessRegionOpen: updating row nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
in region .META.,,1 with  with startcode 1248286712265 and server XX.XX.44.185:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 19:58:40,560 DEBUG
org.apache.hadoop.hbase.master.BaseScanner: Current assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
is not valid;  serverInfo: address: XX.XX.45.124:20020, startcode: 1248286712292, load: (requests=0,
regions=49, usedHeap=101, maxHeap=1820), passed startCode: 1248174024124, storedInfo.startCode:
1248286712292
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 19:58:41,823 INFO
org.apache.hadoop.hbase.master.RegionManager: Assigning region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
to XX.XX.44.49:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 19:59:00,842 INFO
org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259:
safeMode=false from XX.XX.44.49:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 20:04:08,569 INFO
org.apache.hadoop.hbase.master.ProcessRegionOpen: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
open on XX.XX.44.49:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22 20:04:08,569 INFO
org.apache.hadoop.hbase.master.ProcessRegionOpen: updating row nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259
in region .META.,,1 with  with startcode 1248292287873 and server XX.XX.44.49:20020
{code}

> Holes in master state change; updated startcode and server go into .META. but catalog
scanner just got old values
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1573
>                 URL: https://issues.apache.org/jira/browse/HBASE-1573
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: 1573-v2.patch, 1573-v3.patch, 1573.patch
>
>
> Here is example of a scan that takes a while because 6k regions acting on stale data
resulting in double assignment of region:
> {code}
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:06,220
INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_OPEN: enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748:
safeMode=false from XX.XX.45.121:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:06,220
INFO org.apache.hadoop.hbase.master.RegionServerOperation: enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748
open on XX.XX.45.121:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:06,220
INFO org.apache.hadoop.hbase.master.RegionServerOperation: updating row enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748
in region .META.,,1 with  with startcode !?B and server XX.XX.45.121:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:06,397
DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current assignment of enwikibase,Cwj0sehVeEbnrUDR_j0xok==,1245604540748
is not valid;  Server 'XX.XX.44.95:20020' unknown.
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:06,582
INFO org.apache.hadoop.hbase.master.RegionManager: Assigning region enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748
to XX.XX.45.97:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:09,587
INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_PROCESS_OPEN: enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748:
safeMode=false from XX.XX.45.97:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:12,614
INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_OPEN: enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748:
safeMode=false from XX.XX.45.97:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:13,549
INFO org.apache.hadoop.hbase.master.RegionServerOperation: enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748
open on XX.XX.45.97:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22 10:56:13,549
INFO org.apache.hadoop.hbase.master.RegionServerOperation: updating row enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748
in region .META.,,1 with  with startcode !?? and server XX.XX.45.97:20020
> {code}
> We've just updated the server info in the master because of the region open message but
the scan sees old info in the .META. table though .META. was just updated.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message