hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1886) move table and column family attributes to ZK to make them live
Date Mon, 19 Oct 2009 17:29:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767393#action_12767393
] 

Jonathan Gray commented on HBASE-1886:
--------------------------------------

Agree with JD on all points.  Don't think this should be part of 0.20 branch, methods belong
in HBaseAdmin, and we should be using ZK watchers properly not artificially synchronizing.

As for layout in ZK... Do we want this very hierarchical layout with a high number of total
nodes and where nodes are actually storing a good bit of the data (attribute names, family
names, etc)... 

Another thought is to use JSON to store this stuff in fewer total nodes.  Dealing with lots
of hierarchy can make it much more confusing to interact with this stuff.  Rather than just
watching for changed data on a node (for example, change in table configuration) you have
to watch for a change in the children of a node.  The changed children event doesn't actually
tell you what has changed, you have to do some kind of DIFF to know what's changed.  To monitor
for changes to a single attribute of a single family, you'd have to have watchers for every
single attribute value node.  As long as we have _some_ hierarchy, so there isn't just one
huge JSON string, then we could make usage/monitoring much easier.

Sorry if above is confusing... But ideally we'd want to be able to actually monitor for live
changes to any of this stuff.  The more hierarchy and nodes we have (and the more we have
to monitor for changes in the children of a node VS changes in the data of a node), the more
difficult it will be.

{noformat}
/hbase
  /table
    /<table-name>({'json':'table attributes'})
      /<family-name>({'json':'family attributes'})
{noformat}

Not sure how big of a difference that actually makes, but I've built a few systems around
ZK and have definitely found reducing hierarchy and keeping stuff together in JSON has kept
things easier to manage (when you don't have a reason to read the dimensions individually,
so no downside to aggregating into a json string).

Good stuff, Andrew.

> move table and column family attributes to ZK to make them live
> ---------------------------------------------------------------
>
>                 Key: HBASE-1886
>                 URL: https://issues.apache.org/jira/browse/HBASE-1886
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 0.20.2, 0.21.0
>
>         Attachments: live_attributes.patch
>
>
> This is a first cut at moving table and column family attributes up to ZK where they
can be modified any time without requiring a disable/enable table cycle. Adds methods to ZooKeeperWrapper
for support server side. Adds methods to HTableInterface for client side access. Client side
talks directly to ZK. 
> The new hierarchy for attributes in ZK is like:
> {noformat}
> /hbase
>   /table
>     /<table-name>
>       /attr
>         /<table-attr-name>
>         /<family-name>
>           /<family-attr-name>
> {noformat}
> As before attribute names and values are byte[], but now that they are hosted in ZK,
the path delimiter ('/') cannot be used.
> Attached patch compiles but has not been tested. I put it up for a concept review. Next
step before this is functional is to deprecate the attribute related methods on HTD and HCD
and chase through the warnings to find all users who should be updated to use the ZK wrapper
or new HTable methods instead. And, of course, then add test cases that successfully complete.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message