hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: Why ScannerTimeoutException is not handled in TableInputFormat?
Date Tue, 06 Mar 2012 23:25:55 GMT
Hi Jan,

What version were you on before and what version are you on currently?

The HBASE-4269 tried to restore the semantics changed in HBASE-4196 from
0.90.3 to 0.90.5. If we caught the exception we may skip records if the
timeout exceptions are skipped which is undesirable.

It might make sense to change Scanner Timeout to be derived from a
different exception but this will take a bit of digging and testing to
verify (DNRIOEx is used in about 40 places)

Jon.

On Fri, Feb 24, 2012 at 5:08 AM, Jan Lukavsk√Ĺ
<jan.lukavsky@firma.seznam.cz>wrote:

> Hi all,
>
> patch to HBASE-4269 removed handling of ScannerTimeoutException from
> TableRecordReader. Is there a reason for this? We are now seeing a lot of
> ScannerTimeoutExceptions in our jobs, which were previously handled. The
> problem is caused by ScannerTimeoutException being derived from
> DoNotRetryIOException (why's that?). Would it be safe to just derive this
> exception from IOException? And if not, should the TableRecordReader
> explictly handle this exception? The same problem may arise with
> LeaseException, which is also derived from DoNotRetryIOException. Both of
> these exception may have the same cause and in my understanding can be
> handled in the RecordReader without any harm. Or am I wrong?
>
> Thanks for answer,
>  Jan
>
>


-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message