hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5162) Basic client pushback mechanism
Date Tue, 04 Nov 2014 21:58:36 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196914#comment-14196914
] 

Hadoop QA commented on HBASE-5162:
----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12679302/hbase-5162-trunk-v2.patch
  against trunk revision .
  ATTACHMENT ID: 12679302

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 7 new or modified
tests.

    {color:red}-1 javac{color}.  The applied patch generated 103 javac compiler warnings (more
than the trunk's current 102 warnings).

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 warning messages.

                {color:red}-1 checkstyle{color}.  The applied patch generated 3791 checkstyle
errors (more than the trunk's current 3782 errors).

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +      return RpcRetryingCallerFactory.instantiate(conf, this.interceptor, this.getStatisticsTracker());
+    String className = conf.get(BACKOFF_POLICY_CLASS, ExponentialClientBackoffPolicy.class.getName());
+              new java.lang.String[] { "Index", "Result", "Exception", "ServiceResult", "LoadStats",
});

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/checkstyle-aggregate.html

                Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//artifact/patchprocess/patchJavadocWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/11583//console

This message is automatically generated.

> Basic client pushback mechanism
> -------------------------------
>
>                 Key: HBASE-5162
>                 URL: https://issues.apache.org/jira/browse/HBASE-5162
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.92.0
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jesse Yates
>             Fix For: 1.0.0
>
>         Attachments: hbase-5162-trunk-v0.patch, hbase-5162-trunk-v1.patch, hbase-5162-trunk-v2.patch,
java_HBASE-5162.patch
>
>
> The current blocking we do when we are close to some limits (memstores over the multiplier
factor, too many store files, global memstore memory) is bad, too coarse and confusing. After
hitting HBASE-5161, it really becomes obvious that we need something better.
> I did a little brainstorm with Stack, we came up quickly with two solutions:
>  - Send some exception to the client, like OverloadedException, that's thrown when some
situation happens like getting past the low memory barrier. It would be thrown when the client
gets a handler and does some check while putting or deleting. The client would treat this
a retryable exception but ideally wouldn't check .META. for a new location. It could be fancy
and have multiple levels of pushback, like send the exception to 25% of the clients, and then
go up if the situation persists. Should be "easy" to implement but we'll be using a lot more
IO to send the payload over and over again (but at least it wouldn't sit in the RS's memory).
>  - Send a message alongside a successful put or delete to tell the client to slow down
a little, this way we don't have to do back and forth with the payload between the client
and the server. It's a cleaner (I think) but more involved solution.
> In every case the RS should do very obvious things to notify the operators of this situation,
through logs, web UI, metrics, etc.
> Other ideas?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message