hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Filter for deprecated mapred APIs
Date Mon, 24 Jan 2011 17:30:48 GMT
 Want to try with one of the filters we ship with to see if it has
same issue?  If so, please file an issue.  Thats pretty serious bug.

Thanks,
St.Ack

2011/1/24 Yifeng Jiang <yifeng.jiang@mail.rakuten.co.jp>:
> Hi,
>
> I has a MyFilter class extends FilterBase, and a MyInputFormat extends
> hbase.mapred.TableInputFormat, the deprecated mapred APIs.
>
> It seems that the filter will not be invoke when there are only a few
> data in the table.
>
> This is the code in my InputFormat's configure method.
>
> String startDate = job.get("key.startdate");
> String endDate = job.get("key.enddate");
> Filter filter = new MyFilter(sdf.parse(startDate), sdf.parse(endDate));
> setRowFilter(filter);
>
> And the job setup code.
>
> job.setInputFormat(MyInputFormat.class);
> FileInputFormat.addInputPaths(job, "my_table_in_hbase");
> job.set(TableInputFormat.COLUMN_LIST, "data:");
>
> When I put lots of data (> 500,000) in the table, the filter works well,
> but when I put only a few data (<100) in the table, the filter does not
> function any all, and the log in the filter has no output either.
>
> Is there something wrong in my code?
> I'm using HBase0.90.0 and have put MyFilter in HBase's classpath.
>
> Thanks.
>
> --
> Yifeng Jiang
>
>

Mime
View raw message