hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2117) Simple check on the master overview page if the number of currently running regionservers is unchanged.
Date Tue, 12 Jan 2010 17:45:55 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799286#action_12799286
] 

stack commented on HBASE-2117:
------------------------------

Thanks for the patch.

While its true that the regionserver file will usually list all members of a cluster, on occasion
it may lag what is actually up and running: e.g. up on ec2, I believe the regionserver file
is not in alignment with what servers make up the cluster.  So, rather than identifying the
mismatch between actual cluster members and content of the regionserver file as an ERROR,
rather, I'd suggest it should be couched as a gentle hint that the two are not in alignment.
 Admins should be able to easily ignore this message for the case where they have intentionally
misaligned the two.  On the other hand, a gentle prompting could be handy reminder that a
newly added regionserver needs to be added to the regionserver file if the admin wants it
started as part of general cluster restart next time around.

> Simple check on the master overview page if the number of currently running regionservers
is unchanged.
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-2117
>                 URL: https://issues.apache.org/jira/browse/HBASE-2117
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: master, regionserver
>    Affects Versions: 0.20.2
>            Reporter: Ferdy
>         Attachments: HBASE-2117.patch
>
>
> Incidentally, it happens that some of our regionservers just stop working. The regionserver
logs show some sort of termination and the affected regionserver is just removed from the
master page. Besides the actual problem of the termination, what I was missing was some sort
of warning (from either running client code or the master page) that some regionservers are
having trouble.
> It seems like the Master is ok with the fact that a regionserver suddenly decides to
stop. The result is that the clients depending on the data in Hbase will be presented an incomplete
data set, at least as long as the failing regions are not re-assigned yet. In order to have
this monitored, I decided to create a patch that exposes an extra piece of information on
the master page. An 'OK:' is presented if the current number of regionservers is unchanged
since the start of the processes. An 'ERROR:' is shown whenever the current number is not
the same. What the master page does is reading the 'regionservers' file once, and remember
the number of slaves so that is can be used in the check. (So afterwards changes to this file
are not supported).
> Perhaps this is not the right way of doing things. Please let me know if there are any
existing solutions for these issues.
> I will attach a patch right away.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message