hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guanghao Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19035) Miss metrics when coprocessor use region scanner to read data
Date Thu, 23 Nov 2017 02:16:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16263689#comment-16263689
] 

Guanghao Zhang commented on HBASE-19035:
----------------------------------------

The branch-1 HADOOP QA build timeout.... See https://builds.apache.org/job/PreCommit-HBASE-Build/9963/console
It take 6 hours to run the hbase-server test, then timeout.

09:19:11 cd /testptch/hbase/hbase-server
09:19:11 mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hbase-branch-1-patch-1 -DHBasePatchProcess
-PrunAllTests -Dtest.exclude.pattern=**/TestClassFinder.java,**/client.TestGet.java,**/master.cleaner.TestReplicationZKNodeCleaner.java,**/snapshot.TestExportSnapshot.java,**/master.TestAssignmentManagerMetrics.java,**/client.TestShell.java,**/master.assignment.TestAssignmentManager.java,**/master.assignment.TestMergeTableRegionsProcedure.java,**/client.TestAsyncTableGetMultiThreaded.java,**/security.visibility.TestVisibilityLabelsOnNewVersionBehaviorTable.java,**/master.balancer.TestFavoredStochasticLoadBalancer.java,**/client.TestBlockEvictionFromClient.java,**/security.access.TestCoprocessorWhitelistMasterObserver.java,**/master.TestRollingRestart.java,**/client.TestTableSnapshotScanner.java,**/client.TestAsyncTableScanAll.java,**/rsgroup.TestRSGroups.java,**/quotas.TestMasterSpaceQuotaObserver.java,**/replication.TestReplicationKillSlaveRS.java,**/replication.TestReplicationDroppedTables.java,**/quotas.TestQuotaThrottle.java,**/client.TestReplicasClient.java,**/snapshot.TestMobRestoreFlushSnapshotFromClient.java,**/client.locking.TestEntityLocks.java,**/client.TestScannersFromClientSide.java,**/quotas.TestSpaceQuotasWithSnapshots.java,**/client.TestMobSnapshotCloneIndependence.java,**/client.TestReplicaWithCluster.java,**/quotas.TestQuotaAdmin.java,**/TestCheckTestClasses.java,**/master.procedure.TestEnableTableProcedure.java,**/regionserver.TestSplitTransactionOnCluster.java,**/client.TestMultiParallel.java,**/client.TestSizeFailures.java,**/client.TestRestoreSnapshotFromClientWithRegionReplicas.java,**/client.TestAdmin2.java,**/regionserver.TestHRegion.java,**/master.procedure.TestTruncateTableProcedure.java,**/security.visibility.TestVisibilityLabelsWithACL.java,**/master.TestWarmupRegion.java,**/snapshot.TestSecureExportSnapshot.java,**/io.encoding.TestLoadAndSwitchEncodeOnDisk.java,**/master.procedure.TestServerCrashProcedure.java,**/client.replication.TestReplicationAdminWithClusters.java,**/client.TestHCM.java,**/client.replication.TestReplicationAdminWithTwoDifferentZKClusters.java,**/TestJMXListener.java,**/trace.TestHTraceHooks.java,**/replication.TestReplicationSyncUpTool.java,**/client.TestMultiRespectsLimits.java,**/regionserver.TestCompactionInDeadRegionServer.java,**/client.TestAsyncTableAdminApi.java,**/snapshot.TestMobSecureExportSnapshot.java,**/replication.TestMasterReplication.java,**/client.TestAsyncSnapshotAdminApi.java,**/master.assignment.TestAssignmentOnRSCrash.java,**/regionserver.wal.TestAsyncLogRolling.java,**/replication.TestReplicationSmallTests.java,**/snapshot.TestMobFlushSnapshotFromClient.java,**/quotas.TestSnapshotQuotaObserverChore.java,**/TestAcidGuarantees.java,**/master.assignment.TestSplitTableRegionProcedure.java,**/replication.regionserver.TestTableBasedReplicationSourceManagerImpl.java,**/TestZooKeeper.java,**/fs.TestBlockReorder.java,**/client.TestCloneSnapshotFromClient.java,**/security.token.TestTokenAuthentication.java,**/coprocessor.TestRegionObserverInterface.java,**/regionserver.TestFSErrorsExposed.java,**/client.TestMetaWithReplicas.java,**/client.TestFromClientSideWithCoprocessor.java,**/master.TestDistributedLogSplitting.java,**/TestServerSideScanMetricsFromClientSide.java,**/regionserver.TestPerColumnFamilyFlush.java,**/client.TestMobCloneSnapshotFromClient.java,**/TestRegionRebalancing.java,**/security.visibility.TestVisibilityLabelsWithDeletes.java,**/master.procedure.TestMasterFailoverWithProcedures.java,**/master.cleaner.TestHFileCleaner.java
clean test -fae > /testptch/patchprocess/patch-unit-hbase-server.txt 2>&1
15:30:49 Build timed out (after 420 minutes). Marking the build as failed.
15:30:50 Build was aborted

> Miss metrics when coprocessor use region scanner to read data
> -------------------------------------------------------------
>
>                 Key: HBASE-19035
>                 URL: https://issues.apache.org/jira/browse/HBASE-19035
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Guanghao Zhang
>            Assignee: Guanghao Zhang
>             Fix For: 2.0.0-beta-1
>
>         Attachments: HBASE-19035.branch-1.001.patch, HBASE-19035.branch-1.patch, HBASE-19035.branch-1.patch,
HBASE-19035.branch-1.patch, HBASE-19035.branch-1.patch, HBASE-19035.master.001.patch, HBASE-19035.master.002.patch,
HBASE-19035.master.003.patch, HBASE-19035.master.003.patch
>
>
> Region interface is exposed to coprocessor. So coprocessor use getScanner to get a region
scanner to read data. But the scan metrics was only updated in region server level. So we
will miss some scan metrics for the read from coprocessor.
> || Region Operation || When to update requests metric ||
> | get | update read metric in nextRaw() |
> | put | update write metric in batchMutate() |
> | delete | update write metric in batchMutate() |
> | increment | update read metric by get() and  update write metric in doDelta() |
> | append | update read metric by get() and  update write metric in doDelta() |
> | mutateRow | update write metric in processRowsWithLocks() |
> | mutateRowsWithLocks | update write metric in processRowsWithLocks() |
> | batchMutate | update write metric in batchMutate() |
> | checkAndMutate | update read metric by get() and  update write metric by mutateRow()
|
> | checkAndRowMutate | update read metric by get() and  update write metric by doBatchMutate()
|
> | processRowsWithLocks | update write metric in processRowsWithLocks() |
> 1. Move read requests to region level. Because RegionScanner exposed to CP.
> 2. Update write requests count in processRowsWithLocks. This was missed in previous implemenation,
too.
> 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed by region's
readRequestsCount and writeRequestsCount.
> Upload to review board: https://reviews.apache.org/r/63579/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message