hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Strange performance behavior of SingleValColumnFilter
Date Fri, 21 Oct 2011 22:49:36 GMT
We have been doing some performance testing on HBase filters. One outcome was HBASE-4626 (which
I fixed and committed yesterday night).

Now we found a rather strange behavior with SingleColumnValueFilter. On our test cluster it
is 10x slower than ValueFilter, even when we restrict the scan to just the one column we are
filtering on and set filterIfMissing to true.
We are not seeing that with HBase in local mode, which points to some additional activity
on the FS, which in HDFS would be slow compared to a local FS.


Indeed it turns out the problem goes away when we replace all NEXT_ROW with SKIP in SingleColumnValueFilter.filterKeyValue
the performance is *much* better (on par with ValueFilter).


We're using something pretty close to trunk for our tests.
The tables are pretty wide, only one version of each cells (and freshly major compacted).


I do not know this part of the code that well (yet) and was wondering if somebody could chime
in. Maybe this is related to HFileV2?

I do recall there was something done to optimize reseeks. Generally I would have expected
NEXT_ROW to be a major performance improvement.

Any ideas, comments, pointers?

Thanks.

-- Lars


Mime
View raw message