hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12911) Client-side metrics
Date Sat, 05 Sep 2015 04:22:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731790#comment-14731790
] 

Hadoop QA commented on HBASE-12911:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12754302/0001-HBASE-12911-Client-side-metrics.patch
  against master branch at commit 2969093b5b39cb950d8710cfffa7e55484d40259.
  ATTACHMENT ID: 12754302

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 36 new or modified
tests.

    {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions
(2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 protoc{color}.  The applied patch does not increase the total number of
protoc compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

                {color:red}-1 checkstyle{color}.  The applied patch generated 1843 checkstyle
errors (more than the master's current 1834 errors).

    {color:green}+1 findbugs{color}.  The patch does not introduce any  new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

    {color:red}-1 site{color}.  The patch appears to cause mvn post-site goal to fail.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.util.TestProcessBasedCluster
                  org.apache.hadoop.hbase.mapreduce.TestImportExport
                  org.apache.hadoop.hbase.client.TestClientTimeouts
                  org.apache.hadoop.hbase.regionserver.TestWALLockup

     {color:red}-1 core zombie tests{color}.  There are 8 zombie test(s): 	at org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence.testOnlineSnapshotMetadataChangesIndependent(TestMobSnapshotCloneIndependence.java:147)
	at org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence.testOnlineSnapshotMetadataChangesIndependent(TestMobSnapshotCloneIndependence.java:147)
	at org.apache.hadoop.hbase.replication.TestPerTableCFReplication.testPerTableCFReplication(TestPerTableCFReplication.java:291)
	at org.apache.hadoop.hbase.replication.TestPerTableCFReplication.testPerTableCFReplication(TestPerTableCFReplication.java:291)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testNotInMetaHole(TestHBaseFsck.java:1356)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testMissingRegionInfoQualifier(TestHBaseFsck.java:2472)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsck(TestHBaseFsck.java:207)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testNotInHdfsWithReplicas(TestHBaseFsck.java:1440)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testDupeStartKey(TestHBaseFsck.java:705)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testContainedRegionOverlap(TestHBaseFsck.java:979)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testHbckFixOrphanTable(TestHBaseFsck.java:563)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testDupeRegion(TestHBaseFsck.java:877)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testNoHdfsTable(TestHBaseFsck.java:1508)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testCheckTableLocks(TestHBaseFsck.java:2620)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineCorruptMobFile(TestHBaseFsck.java:2288)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testNoVersionFile(TestHBaseFsck.java:1544)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testCleanUpDaughtersNotInMetaAfterFailedSplit(TestHBaseFsck.java:1315)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testHDFSRegioninfoMissing(TestHBaseFsck.java:1227)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testFixMetaNotWorkingWithNoHdfsChecking(TestHBaseFsck.java:2058)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testHbckWithFewerReplica(TestHBaseFsck.java:759)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testRegionDeployedNotInHdfs(TestHBaseFsck.java:1949)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testRegionBoundariesCheck(TestHBaseFsck.java:2824)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testLingeringSplitParent(TestHBaseFsck.java:1726)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testHbckWithExcessReplica(TestHBaseFsck.java:783)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testReadOnlyProperty(TestHBaseFsck.java:2843)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineCorruptHFile(TestHBaseFsck.java:2248)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingHFile(TestHBaseFsck.java:2373)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testFixHdfsHolesNotWorkingWithNoHdfsChecking(TestHBaseFsck.java:2123)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testCoveredStartKey(TestHBaseFsck.java:1157)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsckClean(TestHBaseFsck.java:516)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testNotInMetaHole(TestHBaseFsck.java:1356)
	at org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsckClean(TestHBaseFsck.java:516)
	at org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenSlaveClusterDoesntHaveTable(TestReplicationAdminWithClusters.java:65)
	at org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testDisableAndEnableReplication(TestReplicationAdminWithClusters.java:123)
	at org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationForNonExistingTable(TestReplicationAdminWithClusters.java:148)
	at org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenTableNameAsNull(TestReplicationAdminWithClusters.java:160)
	at org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenTableDescriptorIsNotSameInClusters(TestReplicationAdminWithClusters.java:97)
	at org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenReplicationNotEnabled(TestReplicationAdminWithClusters.java:83)
	at org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenSlaveClusterDoesntHaveTable(TestReplicationAdminWithClusters.java:65)
	at org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenReplicationNotEnabled(TestReplicationAdminWithClusters.java:83)

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/15424//testReport/
Release Findbugs (version 2.0.3) 	warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/15424//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/15424//artifact/patchprocess/checkstyle-aggregate.html

                Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/15424//console

This message is automatically generated.

> Client-side metrics
> -------------------
>
>                 Key: HBASE-12911
>                 URL: https://issues.apache.org/jira/browse/HBASE-12911
>             Project: HBase
>          Issue Type: New Feature
>          Components: Client, Operability, Performance
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>             Fix For: 2.0.0, 1.3.0
>
>         Attachments: 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch,
0001-HBASE-12911-Client-side-metrics.patch, am.jpg, client metrics RS-Master.jpg, client metrics
client.jpg, conn_agg.jpg, connection attributes.jpg, ltt.jpg, standalone.jpg
>
>
> There's very little visibility into the hbase client. Folks who care to add some kind
of metrics collection end up wrapping Table method invocations with {{System.currentTimeMillis()}}.
For a crude example of this, have a look at what I did in {{PerformanceEvaluation}} for exposing
requests latencies up to {{IntegrationTestRegionReplicaPerf}}. The client is quite complex,
there's a lot going on under the hood that is impossible to see right now without a profiler.
Being a crucial part of the performance of this distributed system, we should have deeper
visibility into the client's function.
> I'm not sure that wiring into the hadoop metrics system is the right choice because the
client is often embedded as a library in a user's application. We should have integration
with our metrics tools so that, i.e., a client embedded in a coprocessor can report metrics
through the usual RS channels, or a client used in a MR job can do the same.
> I would propose an interface-based system with pluggable implementations. Out of the
box we'd include a hadoop-metrics implementation and one other, possibly [dropwizard/metrics|https://github.com/dropwizard/metrics].
> Thoughts?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message