hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4908) HBase cluster test tool (port from 0.89-fb)
Date Wed, 30 Nov 2011 23:53:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160476#comment-13160476

Phabricator commented on HBASE-4908:

stack has commented on the revision "[jira] [HBASE-4908] HBase cluster test tool (port from

  Kinda close to PE but I suppose different enough.  This can give constant update on rates
of read and write.  Could add that to PE.  Also, can do load+read at same time.  Would have
to do that in different threads in PE.  Has more variable load too.

  src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:17 is this the right
package M?   util?
  src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:47 A bit of class doc
on what it is might help.

  Might also just point them at usage it seems since that looks like it might be pretty good.
  src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:52 No biggie but these
data member names with a trailing underscore a bit silly (smile)
  src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:58 I suppose its fine
this is hard-coded for now.
  src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:92 No biggie: check cols
is at least 3 in size so we can return 'nicer' message if less than 3
  src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:103 This hard coding
should be explained somewhere?  Maybe in class comment?  The 20/1.  Or in usage.
  src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:204 Yeah, I think these
'features' need to bubble it up to the class comment so folks can see whats in here w/o having
to read code.
  src/test/java/org/apache/hadoop/hbase/manual/HBaseClusterTest.java:275 Hmm... maybe this
is good enough when it prints out for figuring whats in here?
  src/test/java/org/apache/hadoop/hbase/manual/RestartMetaTest.java:36 Class comment
  src/test/java/org/apache/hadoop/hbase/manual/RestartMetaTest.java:17 ditto on package. 
Should this be in under src/test or under src/main?  Maybe package should be tool rather than
  src/test/java/org/apache/hadoop/hbase/manual/RestartMetaTest.java:152 There is a lot of
overlap w/ previous test here, the HBaseClusterTest tool?
  src/test/java/org/apache/hadoop/hbase/manual/utils/DataGenerator.java:24 comment on what
this does?
  src/test/java/org/apache/hadoop/hbase/manual/utils/DisplayFormatUtils.java:24 This class
not needed?  Use http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/util/StringUtils.html#humanReadableInt(long)
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:38 Class comment.  HBaseUtils
is a pretty generic name; says nothing about content.
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:53 Remove
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:54 Why not just let this
out?  Should at least say in javadoc that it can return a null?
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:59 This method does not
look like it belongs in a generic hbaseutils class.  It looks like it belongs to the testing
tool that makes up the bulk of this patch.   Doesn't seem like method name describes properly
what it does either.  Why not let out exceptions rather than catch and log.
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:92 HSA is deprecated.
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:96 Use MetaReader? e.g:

  public static Pair<HRegionInfo, ServerName> getRegion(
        CatalogTracker catalogTracker, byte [] regionName)
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:99 Odd name for this
method and it also looks a bit dodgy returning an HBC but it only has zk location set into
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:111 This looks like it
belongs in Keying class under util?  Or in HBaseSplitter?
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:124 ditto
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:139 This should be in
Bytes (is it not already?)
  src/test/java/org/apache/hadoop/hbase/manual/utils/HBaseUtils.java:151 ditto
  src/test/java/org/apache/hadoop/hbase/manual/utils/HdfsAppender.java:36 Does this belong
in an hbase test class?
  src/test/java/org/apache/hadoop/hbase/manual/utils/KillProcessesAndVerify.java:33 class
  src/test/java/org/apache/hadoop/hbase/manual/utils/KillProcessesAndVerify.java:119 Let these
out rather than just print them?
  src/test/java/org/apache/hadoop/hbase/manual/utils/KillProcessesAndVerify.java:152 What
keys are being passed here?
  src/test/java/org/apache/hadoop/hbase/manual/utils/MultiThreadedAction.java:31 Class comment
  src/test/java/org/apache/hadoop/hbase/manual/utils/MultiThreadedReader.java:156 Remove
Class comment.  Is this doing what minihbasecluster does?
Not sure this a good idea.


> HBase cluster test tool (port from 0.89-fb)
> -------------------------------------------
>                 Key: HBASE-4908
>                 URL: https://issues.apache.org/jira/browse/HBASE-4908
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Mikhail Bautin
>            Assignee: Mikhail Bautin
>         Attachments: D549.1.patch
> Porting one of our HBase cluster test tools (a single-process multi-threaded load generator
and verifier) from 0.89-fb to trunk.
> I cleaned up the code a bit compared to what's in 0.89-fb, and discovered that it has
some features that I have not tried yet (some kind of a kill test, and some way to run HBase
as multiple processes on one machine).
> The main utility of this piece of code for us has been the HBaseClusterTest command-line
tool (called HBaseTest in 0.89-fb), which we usually invoke as a load test in our five-node
dev cluster testing, e.g.:
> hbase org.apache.hadoop.hbase.manual.HBaseTest -load 1000000000:50:100:20 -tn load_test
-read 1:1000000000:50:20 -zk <zk_quorum> -bloom ROWCOL -compression GZIP
> I will be using this code to load-test the delta encoding patch and making fixes, but
I am submitting the patch for early feedback. I will probably try out its other functionality
and comment on how it works.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message