hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: InternalScanner next(..) methods
Date Tue, 11 Dec 2012 06:56:50 GMT
The offending method here is:

  public void filterRow(List<KeyValue> kvs);
on Filter.java

But even if we changed that to accept a stream of KVs, how are we going to make sure the filter
code does not hold on to the KVs it received?
As long as we allow custom code in filter we cannot reuse the memory backing the KVs, unless
we force the filter code to make a copy of any KV it wants to hold on (which would make the
above method extremely expensive).

Of our pre-canned filters only DependentColumnFilter implements this method.

-- Lars

 From: Stack <stack@duboce.net>
To: HBase Dev List <dev@hbase.apache.org>; lars hofhansl <lhofhansl@yahoo.com>

Sent: Monday, December 10, 2012 12:26 PM
Subject: Re: InternalScanner next(..) methods
On Sun, Dec 9, 2012 at 12:52 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:

> This method specifically only works when this is a heap of StoreScanners
> (i.e. on the RegionScanner level), which is very confusing (to me anyway).
> Maybe we should have two separate KeyValueHeap implementation to make it
> less confusing.
> The list here comprises KVs for the same row key. These KVs need to be
> collected together so that Filters can operate on entire rows.
We should change Filter Interface instead giving it a stream rather than
"whole row"?

> I just looked at that code this week. We need to fix this stuff. :)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message