hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1502) Remove need for heartbeats in HBase
Date Wed, 16 Feb 2011 22:53:24 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-1502:

    Attachment: 1502-4.txt

Patch does not compile yet.  Still have shutdown and split to do.  Here is what I have
so far.

Removing  heartbeats has brought on a refactoring.  A couple of our fundamental types
make little sense in absence of heartbeating.  We've also been abusing core types such
as HServerAddress serializing it up into zookeeper, a problematic behavior given deserializing
will bring on a resolve.

An HServerAddress is a Writable InetSocketAddress.  We should just be using ISA.  Passing
an HSA that will resolve when deserialized on other side is something we need to move
away from.  Lets just use ISA from here on out.  HSA was not removed because its in public
API in a few places; it shouldn't be.

HServerInfo has deprecated and removed from all locations but where its part of public API.
It hosts an HServerAddress, HServerLoad, and webuiport.  It was mainly passed by the regionserver
heartbeating.  Now we don't heartbeat, passing load in HServerInfo doesn't make sense.  Also
the webuiport needs to be obtained elsewhere than on heartbeat (its up in zk in the rs znode
Finally, the passing of an HServerAddress inside the HServerInfo is problematic (See above).

HServerLoad has been removed.  It was passed inside HServerInfo on each heartbeat.
TODO: replace with alternative.  I thought of writing 'load' to zk every minute or so but
that could get hefty if lots of servers.  Still might go this route.  Wanted to do it w/
JSON but HSL wasn't easy to serialize with its aggregating and internal RegionLoad.

Patch includes new class used identifying servers.  Its a formalization of our special
'servername' String.  The class is called ServerName.  Its not a Writable.  This class is
used near everywhere in place of HSA and/or HSI identifying servers.  A ServerName can be
made from the server + startcode columns in .META.  Its the name of the RS znode in zookeeper.
A bunch of messing with hostname and port only has been removed.

Changes CP Interfaces to use ServerName instead of HSI.

Changes the HMasterRegionInterface to remove the heartbeat.  Upped this interfaces
version number.

Removed from ZKUtil all the getAddress methods.

Redid the master join cluster code.  Previous we relied on HServerInfo on server
registration but this is not available any more.  Instead scrape .META. and RIT
to figure whether failover or fresh cluster start.

Changd the way we do our address in Master and RS. Removed HSI and HSA.  Using
InetSocketAddress instead.  Cleaned a bunch of crud from RS in particular.
Made this code same in master and rs.

RegionServer now registers in ZK only after registering first with master.
It will use the hostname that the master sees the regionserver in its znode
name.  RS is careful to use the hostname the master passed us writing .META.
(doesn't use the RS hostname -- there is no RS hostname kept any more other than
what is in the RS ISA).

RegionServerTracker now keeps running list of servers.  Also has znode content
which currently is the webuiport only.

> Remove need for heartbeats in HBase
> -----------------------------------
>                 Key: HBASE-1502
>                 URL: https://issues.apache.org/jira/browse/HBASE-1502
>             Project: HBase
>          Issue Type: Task
>            Reporter: Nitay Joffe
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.92.0
>         Attachments: 1502-4.txt, 1502-v2.txt, 1502.txt
> HBase currently uses heartbeats between region servers and the master, piggybacking information
on them when it can. This issue is to investigate if we can get rid of the need for those
using ZooKeeper events.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message