hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5757) TableInputFormat should handle as many errors as possible
Date Mon, 21 May 2012 17:10:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280285#comment-13280285
] 

Hadoop QA commented on HBASE-5757:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12528448/5757-trunk-v2.txt
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 hadoop23.  The patch compiles against the hadoop 0.23.x profile.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 33 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.coprocessor.TestClassLoading
                  org.apache.hadoop.hbase.replication.TestReplication
                  org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
                  org.apache.hadoop.hbase.regionserver.wal.TestHLog
                  org.apache.hadoop.hbase.replication.TestMasterReplication

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1945//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1945//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1945//console

This message is automatically generated.
                
> TableInputFormat should handle as many errors as possible
> ---------------------------------------------------------
>
>                 Key: HBASE-5757
>                 URL: https://issues.apache.org/jira/browse/HBASE-5757
>             Project: HBase
>          Issue Type: Bug
>          Components: mapred, mapreduce
>    Affects Versions: 0.90.6
>            Reporter: Jan Lukavsky
>         Attachments: 5757-trunk-v2.txt, HBASE-5757-trunk-r1341041.patch, HBASE-5757.patch,
HBASE-5757.patch
>
>
> Prior to HBASE-4196 there was different handling of IOExceptions thrown from scanner
in mapred and mapreduce API. The patch to HBASE-4196 unified this handling so that if exception
is caught a reconnect is attempted (without bothering the mapred client). After that, HBASE-4269
changed this behavior back, but in both mapred and mapreduce APIs. The question is, is there
any reason not to handle all errors that the input format can handle? In other words, why
not try to reissue the request after *any* IOException? I see the following disadvantages
of current approach
>  * the client may see exceptions like LeaseException and ScannerTimeoutException if he
fails to process all fetched data in timeout
>  * to avoid ScannerTimeoutException the client must raise hbase.regionserver.lease.period
>  * timeouts for tasks is aready configured in mapred.task.timeout, so this seems to me
a bit redundant, because typically one needs to update both these parameters
>  * I don't see any possibility to get rid of LeaseException (this is configured on server
side)
> I think all of these issues would be gone, if the DoNotRetryIOException would not be
rethrown. -On the other hand, handling errors in InputFormat has disadvantage, that it may
hide from the user some inefficiency. Eg. if I have very big scanner.caching, and I manage
to process only a few rows in timeout, I will end up with single row being fetched many times
(and will not be explicitly notified about this). Could we solve this problem by adding some
counter to the InputFormat?-

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message