hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-3159) Double play of OpenedRegionHandler for a single region; fails second time through and aborts Master
Date Thu, 28 Oct 2010 16:05:48 GMT

     [ https://issues.apache.org/jira/browse/HBASE-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Gray updated HBASE-3159:
---------------------------------

    Attachment: rs_death_on_meta_open_no_root.txt

Another issue during TestRollingRestart.

When RS is opening META, it goes to update ROOT location.

In the code, it seems like it's supposed to wait indefinitely for a ROOT location before proceeding
with the edit for the new meta location.  But in the log it hardly waits at all.  And seemingly
returns a null location, which it should not if there's no root location.

In CT.getCachedConnection(HServerAddress) it looks like we will return null in many different
instances (and not even log that a connection exception happened).  Somehow we're returning
true from the blocking calls waiting on root, but then when we go to verify it doesn't work?

> Double play of OpenedRegionHandler for a single region; fails second time through and
aborts Master
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-3159
>                 URL: https://issues.apache.org/jira/browse/HBASE-3159
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.90.0
>
>         Attachments: hbase-meta-dupe-opened-master-only.txt, hbase-meta-dupe-opened.txt,
master-root-assign-abort.log, rs_death_on_meta_open_no_root.txt, TestRollingRestart-v4.patch
>
>
> Here is master log with annotations: http://people.apache.org/~stack/master.txt
> Region in question is:
> b8827a67a9d446f345095d25e1f375f7
> The running code is doctored in that I've added in a bit of logging -- zk in particular
-- and I've also removed what I thought was a provocation of this condition, reassign inside
in an assign if server has gone away when we try the open rpc (Turns out we have the condition
even w/o this code in place).
> The log starts where the region in question timesout in RIT.
> We assign it to 186.
> Notice how we see 'Handling transition' for this region TWICE.  This means two OpenedRegionHandlers
will be scheduled -- and so the failure to delete a znode already gone.
> As best I can tell, the watcher for this region is triggered once only -- which is odd
because how then the double scheduling of OpenedRegionHandler but also, why am I not seeing
OPENING, OPENING, OPENED and only what I presume is an OPENED?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message