hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maddineni Sukumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16466) HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster with large tables
Date Fri, 28 Apr 2017 01:44:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988027#comment-15988027
] 

Maddineni Sukumar commented on HBASE-16466:
-------------------------------------------

Hi Ted, 

Attached v2 patch with format fixes. 

I ran tests without snapshot and it took 76sec and including snapshot unit tests it took 87
sec, so its just 10sec more time. 
I think if I separate them into  their own tests then we will have unnecessary two clusters
setup burden which will take more time. 

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.hbase.replication.TestReplicationSmallTests
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 76.917 sec - in org.apache.hadoop.hbase.replication.TestReplicationSmallTests


-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.hbase.replication.TestReplicationSmallTests
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 87.247 sec - in org.apache.hadoop.hbase.replication.TestReplicationSmallTests
Results :
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0

I ran just snapshot test and it took 25 sec which includes setup.

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.hbase.replication.TestReplicationSmallTests
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.409 sec - in org.apache.hadoop.hbase.replication.TestReplicationSmallTests
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

154 sec for one test is too much. it might be a bad run. Can you please try again.

> HBase snapshots support in VerifyReplication tool to reduce load on live HBase cluster
with large tables
> --------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-16466
>                 URL: https://issues.apache.org/jira/browse/HBASE-16466
>             Project: HBase
>          Issue Type: Improvement
>          Components: hbase
>    Affects Versions: 0.98.21
>            Reporter: Sukumar Maddineni
>            Assignee: Maddineni Sukumar
>             Fix For: 1.3.1
>
>         Attachments: HBASE-16466.branch-1.3.001.patch, HBASE-16466.v1.patch, HBASE-16466.v2.patch
>
>
> As of now VerifyReplicatin tool is running using normal HBase scanners. If you  want
to run VerifyReplication multiple times on a production live cluster with large tables then
it creates extra load on HBase layer. So if we implement snapshot based support then both
in source and target we can read data from snapshots which reduces load on HBase



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message