hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: ScannerTimeoutException
Date Fri, 02 Jan 2009 17:47:26 GMT
Hello Edward,

[...]
> hbase.regionserver.lease.period is 240000.
> table : 100 rows, one column, the cell is 2 MB.
> So, total size of table entries is 200 MB.
> Each map open a scanner of another table and, doing little
> big calculation.
> How can I avoid ScannerTimeoutException?

You should set hbase.regionserver.lease.period equal to 
whatever you have mapred.task.timeout set to. This way 
scanners will not expire before your tasks are considered
dead. Then you should increase these values in tandem
until your computations complete.

However, I would suggest to you that mappers are intended
to operate on single computational units that are not so
large or involved so the mappers can complete relatively
quickly. Perhaps you should also investigate how to 
separate your data items into smaller discrete units. 

Hope this helps,

   - Andy




      

Mime
View raw message