Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 2 Aug 2013 16:29:48 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12661097.1375352776316.159545.1375460988790@arcas>
In-Reply-To: <JIRA.12661097.1375352776316@arcas>
References: <JIRA.12661097.1375352776316@arcas>
Subject: [jira] [Commented] (HBASE-9108) LoadTestTool need to have a way to
 ignore keys which were failed during write.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-9108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727783#comment-13727783 ] 

Hadoop QA commented on HBASE-9108:
----------------------------------

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12595581/HBASE-9108.patch._trunk
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 9 new or modified tests.

    {color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 1.0 profile.

    {color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 2.0 profile.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6572//console

This message is automatically generated.
                
> LoadTestTool need to have a way to ignore keys which were failed during write. 
> -------------------------------------------------------------------------------
>
>                 Key: HBASE-9108
>                 URL: https://issues.apache.org/jira/browse/HBASE-9108
>             Project: HBase
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.95.0, 0.95.1, 0.94.9, 0.94.10
>            Reporter: gautam
>            Assignee: gautam
>            Priority: Critical
>         Attachments: HBASE-9108.patch._0.94, HBASE-9108.patch._trunk
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> While running the chaosmonkey integration tests, it is found that write sometimes fails when the cluster components are restarted/stopped/killed etc..
> The data key which was being put, using the LoadTestTool, is added to the failed key set, and at the end of the test, this failed key set is checked for any entries to assert failures.
> While doing fail-over testing, it is expected that some of the keys may go un-written. The point here is to validate that whatever gets into hbase for an unstable cluster really goes in, and hence read should be 100% for whatever keys went in successfully.
> Currently LoadTestTool has strict checks to validate every key being written or not. In case any keys is not written, it fails.
> I wanted to loosen this constraint by allowing users to pass in a set of exceptions they expect when doing put/write operations over hbase. If one of these expected exception set is thrown while writing key to hbase, the failed key would be ignored, and hence wont even be considered again for subsequent write as well as read.
> This can be passed to the load test tool as csv list parameter -allowed_write_exceptions, or it can be passed through hbase-site.xml by writing a value for "test.ignore.exceptions.during.write"
> Here is the usage:
> -allowed_write_exceptions "java.io.EOFException,org.apache.hadoop.hbase.NotServingRegionException,org.apache.hadoop.hbase.client.NoServerForRegionException,org.apache.hadoop.hbase.ipc.ServerNotRunningYetException"
> Hence, by doing this the existing integration tests can also make use of this change by passing it as property in hbase-site.xml, as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira