hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4534) A new unit test for lazy seek and StoreScanner in general
Date Tue, 04 Oct 2011 04:37:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119895#comment-13119895
] 

jiraposter@reviews.apache.org commented on HBASE-4534:
------------------------------------------------------



bq.  On 2011-10-04 00:56:39, Lars Hofhansl wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java, line 100
bq.  > <https://reviews.apache.org/r/2155/diff/1/?file=47617#file47617line100>
bq.  >
bq.  >     On many JVMs (including hotspot) this creates a memory barrier.
bq.  >     This might actually cause a measurable slowdown on multi-core machines.
bq.  >     (And maybe I'm just too paranoid).
bq.  >
bq.  
bq.  Mikhail Bautin wrote:
bq.      I could shut off these counters in production, but we already increment AtomicLongs
all over the place to collect performance counters. So if this is a concern, this is probably
best addressed in a separate feature. I don't yet know what the solution would be (some custom
high-performance non-locking counters?) We have also added a lot of per-table/column family
metrics in our internal branch, and that did not seem to hurt performance at all. A more important
point, however, is that if we can cut disk IO, a small amount additional computation is probably
OK. We are already seeing a huge reduction in block cache misses after pushing lazy seek to
production.
bq.      
bq.      Please let me know if you still want me to get rid of these increments in production.
bq.  
bq.  Todd Lipcon wrote:
bq.      We already have a dependency on high-scale-lib, which includes efficient striped
counters. But I agree we shouldn't worry about it in a particular patch, and rather do it
throughout if we see a degredation

Yeah... I am probably overly concerned because Seek and reseek are called pretty frequently.
Let's not worry about it.


- Lars


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2155/#review2297
-----------------------------------------------------------


On 2011-10-04 01:20:14, Mikhail Bautin wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2155/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-10-04 01:20:14)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  A randomized unit test for Gets/Scans (all-row, single-row, multi-row, all-column, single-column,
and multi-column). Also all combinations of Bloom filters and compression (NONE vs GZIP) are
tested. The unit test flushes multiple StoreFiles with disjoint timestamp ranges and runs
various types of queries against them. Currently we are not testing overlapping timestamp
ranges.
bq.  
bq.  
bq.  This addresses bug HBASE-4534.
bq.      https://issues.apache.org/jira/browse/HBASE-4534
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java 3ec1962 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java ede42b2 
bq.    src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java a828846 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java a0d8abf

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWithBloomError.java e86c1c6

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestSeekOptimizations.java PRE-CREATION

bq.  
bq.  Diff: https://reviews.apache.org/r/2155/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  All 1026 unit tests passed.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Mikhail
bq.  
bq.


                
> A new unit test for lazy seek and StoreScanner in general
> ---------------------------------------------------------
>
>                 Key: HBASE-4534
>                 URL: https://issues.apache.org/jira/browse/HBASE-4534
>             Project: HBase
>          Issue Type: Test
>    Affects Versions: 0.94.0
>            Reporter: Mikhail Bautin
>            Assignee: Mikhail Bautin
>
> A randomized unit test for Gets/Scans (all-row, single-row, multi-row, all-column, single-column,
and multi-column). Also all combinations of Bloom filters and compression (NONE vs GZIP) are
tested. The unit test flushes multiple StoreFiles with disjoint timestamp ranges and runs
various types of queries against them. Currently we are not testing overlapping timestamp
ranges.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message