hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5335) Dynamic Schema Configurations
Date Mon, 19 Mar 2012 22:33:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232968#comment-13232968
] 

Phabricator commented on HBASE-5335:
------------------------------------

nspiegelberg has commented on the revision "[jira] [HBASE-5335] Dynamic Schema Config".

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java:753 I looked into this some
more.  We would need to escape more than just single quotes.  There is a function StringEscapeUtils.escapeJavaScript()
in apache.commons.lang that uses the same parsing escapes as Ruby.  However, that requires
us to either ensure client packaging of this dependency or write a custom parser for this.

  practically, single quotes are a mistake.  we offer zero parsing sanitization as is and
just let the RegionServer throw an IllegalArgumentException if the user configures it wrong.
 we need to let the user undo a single quote mistake.  this is currently possible.  value
sanitization is the same as existing functionality
  src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java:760 currently, there is no
javadoc info & the user has no way of knowing he can do this unless he knows about it
from the JIRA.  security through obscurity.
  src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java:367 Already talked with Ted
& Stack about this in previous comments.  A summary:

  Both HTD & HCD have massive redundancies that could be unified beyond this function.
 They are both basically decorated K,V stores.  I don't think unification is very necessary
until we have something like locality groups and need a third K,V store.  Have you traditionally
seen bugs related to HTD & HCD inconsistencies?  I'm just worried that there is time spent
on thinking of how to refactor this code without thinking about whether the denormalized design
is actually hurting us on a practical feature design/debug sense.
  src/main/java/org/apache/hadoop/hbase/util/CompoundConfiguration.java:222-229 3 problems:

  1) If I don't use Configuration.java, then I need to refactor a LOT of the regionserver
code to use the new API

  2) we basically need to override Configuration.getProps or we'd need protected access to
Configuration.properties so we can modify that pointer.  Basically, there are a bunch of functions
in the base Configuration that call getProps() and we need to use our derived version instead
of the base version.

  3) Trying to make a generic implementation that works across all HDFS versions.  I would
like to modify Configuration.properties in HDFS 1.0 to be protected, but I'd still need to
have this code until my patch made it to all the versions we support.

  Configuration.java hasn't changed much, so I don't think this is an issue in practice. 
Do you have another strategy?
  src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java:4292 The 10 sec sleep
is found from TestFromClientSide.compactCFRegion.  I used the same paradigm here and added
poll-waiting to speed up the test.  We need the synchronous compaction feature before we can
remove this (HBASE-2949).

REVISION DETAIL
  https://reviews.facebook.net/D2247

                
> Dynamic Schema Configurations
> -----------------------------
>
>                 Key: HBASE-5335
>                 URL: https://issues.apache.org/jira/browse/HBASE-5335
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>              Labels: configuration, schema
>         Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch
>
>
> Currently, the ability for a core developer to add per-table & per-CF configuration
settings is very heavyweight.  You need to add a reserved keyword all the way up the stack
& you have to support this variable long-term if you're going to expose it explicitly
to the user.  This has ended up with using Configuration.get() a lot because it is lightweight
and you can tweak settings while you're trying to understand system behavior [since there
are many config params that may never need to be tuned].  We need to add the ability to put
& read arbitrary KV settings in the HBase schema.  Combined with online schema change,
this will allow us to safely iterate on configuration settings.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message