hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng Hao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6852) SchemaMetrics.updateOnCacheHit costs too much while full scanning a table with all of its fields
Date Sun, 07 Oct 2012 05:43:03 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471150#comment-13471150
] 

Cheng Hao commented on HBASE-6852:
----------------------------------

I re-ran the scanning tests, with or without the patch attached, still, the patched version
got 10% shorter in entire running time.
The oprofile result of the un-patched version as (top 4):
samples  %        image name               symbol name
-------------------------------------------------------------------------------
54182    14.6977  23960.jo                 void org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.incrNumericMetric(org.apache.hadoop.hbase.io.hfile.BlockType$BlockCategory,
boolean, org.a
pache.hadoop.hbase.regionserver.metrics.SchemaMetrics$BlockMetricType)
  54182    100.000  23960.jo                 void org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.incrNumericMetric(org.apache.hadoop.hbase.io.hfile.BlockType$BlockCategory,
boolean, org
.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics$BlockMetricType) [self]
-------------------------------------------------------------------------------
43949    11.9219  23960.jo                 int org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(byte[],
int, int, byte[], int, int)
  43949    100.000  23960.jo                 int org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(byte[],
int, int, byte[], int, int) [self]
-------------------------------------------------------------------------------
20725     5.6220  23960.jo                 int org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.binarySearchNonRootIndex(byte[],
int, int, java.nio.ByteBuffer, org.apache.hadoop.io
.RawComparator)
  20725    100.000  23960.jo                 int org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.binarySearchNonRootIndex(byte[],
int, int, java.nio.ByteBuffer, org.apache.hadoop.io.RawComparator) [self]
-------------------------------------------------------------------------------
17554     4.7618  23960.jo                 org.apache.hadoop.hbase.io.hfile.HFileBlock org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(long,
long, boolean, boolean, boolean, org.apache.hadoop.hbase.io.hfile.BlockType)
  17554    100.000  23960.jo                 org.apache.hadoop.hbase.io.hfile.HFileBlock org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(long,
long, boolean, boolean, boolean, org.apache.hadoop.hbase.io.hfile.BlockType) [self]

And the oprofile result for patched version as (Top 4):
samples  %        image name               symbol name
-------------------------------------------------------------------------------
53716    11.9679  3683.jo                  int org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.binarySearchNonRootIndex(byte[],
int, int, java.nio.ByteBuffer, org.apache.hadoop.io.RawComparator)
  53716    100.000  3683.jo                  int org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.binarySearchNonRootIndex(byte[],
int, int, java.nio.ByteBuffer, org.apache.hadoop.io.RawComparator) [self]
-------------------------------------------------------------------------------
34921     7.7804  3683.jo                  int org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(byte[],
int, int, byte[], int, int)
  34921    100.000  3683.jo                  int org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(byte[],
int, int, byte[], int, int) [self]
-------------------------------------------------------------------------------
31446     7.0061  3683.jo                  org.apache.hadoop.hbase.io.hfile.HFileBlock org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(long,
long, boolean, boolean, boolean, org.apache.hadoop.hbase.io.hfile.BlockType)
  31446    100.000  3683.jo                  org.apache.hadoop.hbase.io.hfile.HFileBlock org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(long,
long, boolean, boolean, boolean, org.apache.hadoop.hbase.io.hfile.BlockType) [self]
-------------------------------------------------------------------------------
20126     4.4841  3683.jo                  org.apache.hadoop.hbase.regionserver.ScanQueryMatcher$MatchCode
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(org.apache.hadoop.hbase.KeyValue)
  20126    100.000  3683.jo                  org.apache.hadoop.hbase.regionserver.ScanQueryMatcher$MatchCode
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(org.apache.hadoop.hbase.KeyValue)
[self]

Perhaps, the function call itself may costs too much, like the stacks poping / pushing etc.
and the patch just reduces the un-necessary function calls.
                
> SchemaMetrics.updateOnCacheHit costs too much while full scanning a table with all of
its fields
> ------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6852
>                 URL: https://issues.apache.org/jira/browse/HBASE-6852
>             Project: HBase
>          Issue Type: Improvement
>          Components: metrics
>    Affects Versions: 0.94.0
>            Reporter: Cheng Hao
>            Priority: Minor
>              Labels: performance
>             Fix For: 0.94.3, 0.96.0
>
>         Attachments: AtomicTest.java, onhitcache-trunk.patch
>
>
> The SchemaMetrics.updateOnCacheHit costs too much while I am doing the full table scanning.
> Here is the top 5 hotspots within regionserver while full scanning a table: (Sorry for
the less-well-format)
> CPU: Intel Westmere microarchitecture, speed 2.262e+06 MHz (estimated)
> Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00
(No unit mask) count 5000000
> samples  %        image name               symbol name
> -------------------------------------------------------------------------------
> 98447    13.4324  14033.jo                 void org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.updateOnCacheHit(org.apache.hadoop.hbase.io.hfile.BlockType$BlockCategory,
boolean)
>   98447    100.000  14033.jo                 void org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.updateOnCacheHit(org.apache.hadoop.hbase.io.hfile.BlockType$BlockCategory,
boolean) [self]
> -------------------------------------------------------------------------------
> 45814     6.2510  14033.jo                 int org.apache.hadoop.hbase.KeyValue$KeyComparator.compareRows(byte[],
int, int, byte[], int, int)
>   45814    100.000  14033.jo                 int org.apache.hadoop.hbase.KeyValue$KeyComparator.compareRows(byte[],
int, int, byte[], int, int) [self]
> -------------------------------------------------------------------------------
> 43523     5.9384  14033.jo                 boolean org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(org.apache.hadoop.hbase.KeyValue)
>   43523    100.000  14033.jo                 boolean org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(org.apache.hadoop.hbase.KeyValue)
[self]
> -------------------------------------------------------------------------------
> 42548     5.8054  14033.jo                 int org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(byte[],
int, int, byte[], int, int)
>   42548    100.000  14033.jo                 int org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(byte[],
int, int, byte[], int, int) [self]
> -------------------------------------------------------------------------------
> 40572     5.5358  14033.jo                 int org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.binarySearchNonRootIndex(byte[],
int, int, java.nio.ByteBuffer, org.apache.hadoop.io.RawComparator)~1
>   40572    100.000  14033.jo                 int org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.binarySearchNonRootIndex(byte[],
int, int, java.nio.ByteBuffer, org.apache.hadoop.io.RawComparator)~1 [self]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message