hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans" <jdcry...@apache.org>
Subject Re: MR process in endless loop
Date Fri, 19 Sep 2008 17:40:33 GMT
Dru,

Which versions?

Thx

J-D

On Fri, Sep 19, 2008 at 1:38 PM, Dru Jensen <drujensen@gmail.com> wrote:

> I have a MR process that gets stuck in an endless loop.  It looks like the
> same set of keys are being sent to one of the tasks in an endless loop.
> Unfortunately, Its not consistent.  Sometimes it works fine.  Only 1 of the
> 6 MR processes gets in this state and never completes.
> After the disk space is used up on the HFS, the tables become corrupt and I
> can no longer recover them.
>
> The main difference from other MR processes I have is that I added a filter
> to the MR process table scanner by extending the TableInputFormatBase:
>
> public class TableInputFormatColumnFilter extends TableInputFormatBase
> implements JobConfigurable {...}
>
> and then adding a ColumnValueFilter in the configure() as follows:
>
>    ColumnValueFilter rowFilter = new
> ColumnValueFilter(Bytes.toBytes("column_name"),
> ColumnValueFilter.CompareOp.LESS_OR_EQUAL, Bytes.toBytes("column_value"));
>    setRowFilter(rowFilter);
>
> Any ideas what may be causing this?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message