hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4534) A new unit test for lazy seek and StoreScanner in general
Date Tue, 04 Oct 2011 01:14:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119839#comment-13119839
] 

jiraposter@reviews.apache.org commented on HBASE-4534:
------------------------------------------------------



bq.  On 2011-10-04 00:56:39, Lars Hofhansl wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java, line 100
bq.  > <https://reviews.apache.org/r/2155/diff/1/?file=47617#file47617line100>
bq.  >
bq.  >     On many JVMs (including hotspot) this creates a memory barrier.
bq.  >     This might actually cause a measurable slowdown on multi-core machines.
bq.  >     (And maybe I'm just too paranoid).
bq.  >

I could shut off these counters in production, but we already increment AtomicLongs all over
the place to collect performance counters. So if this is a concern, this is probably best
addressed in a separate feature. I don't yet know what the solution would be (some custom
high-performance non-locking counters?) We have also added a lot of per-table/column family
metrics in our internal branch, and that did not seem to hurt performance at all. A more important
point, however, is that if we can cut disk IO, a small amount additional computation is probably
OK. We are already seeing a huge reduction in block cache misses after pushing lazy seek to
production.

Please let me know if you still want me to get rid of these increments in production.


- Mikhail


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2155/#review2297
-----------------------------------------------------------


On 2011-10-03 21:59:42, Mikhail Bautin wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2155/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-10-03 21:59:42)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  A randomized unit test for Gets/Scans (all-row, single-row, multi-row, all-column, single-column,
and multi-column). Also all combinations of Bloom filters and compression (NONE vs GZIP) are
tested. The unit test flushes multiple StoreFiles with disjoint timestamp ranges and runs
various types of queries against them. Currently we are not testing overlapping timestamp
ranges.
bq.  
bq.  
bq.  This addresses bug HBASE-4534.
bq.      https://issues.apache.org/jira/browse/HBASE-4534
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java 3ec1962 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java ede42b2 
bq.    src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java a828846 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java a0d8abf

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWithBloomError.java e86c1c6

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestSeekOptimizations.java PRE-CREATION

bq.  
bq.  Diff: https://reviews.apache.org/r/2155/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Re-running the test suite. Please do not commit yet.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Mikhail
bq.  
bq.


                
> A new unit test for lazy seek and StoreScanner in general
> ---------------------------------------------------------
>
>                 Key: HBASE-4534
>                 URL: https://issues.apache.org/jira/browse/HBASE-4534
>             Project: HBase
>          Issue Type: Test
>    Affects Versions: 0.94.0
>            Reporter: Mikhail Bautin
>            Assignee: Mikhail Bautin
>
> A randomized unit test for Gets/Scans (all-row, single-row, multi-row, all-column, single-column,
and multi-column). Also all combinations of Bloom filters and compression (NONE vs GZIP) are
tested. The unit test flushes multiple StoreFiles with disjoint timestamp ranges and runs
various types of queries against them. Currently we are not testing overlapping timestamp
ranges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message