hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10752) Port HBASE-10270 'Remove DataBlockEncoding from BlockCacheKey' to trunk
Date Thu, 20 Mar 2014 17:15:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941985#comment-13941985
] 

Ted Yu commented on HBASE-10752:
--------------------------------

Patch v3 applies to 0.98
Here is what I tested with 0.98 + patch v3:

1. loaded up new hbase-server.jar
2. scanned an existing table - on rs-status#blockCacheStats UI, observed 0 cache eviction
3. performed schema change to use FAST_DIFF encoding:
{code}
hbase(main):001:0> alter 'testtable', {DATA_BLOCK_ENCODING => 'FAST_DIFF'}
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2014-03-20 17:10:49,893 WARN  [main] conf.Configuration: hbase-site.xml:an attempt to override
final parameter: dfs.support.append;  Ignoring.
2014-03-20 17:10:50,424 WARN  [main] conf.Configuration: hbase-site.xml:an attempt to override
final parameter: dfs.support.append;  Ignoring.
2014-03-20 17:10:50,481 WARN  [main] conf.Configuration: hbase-site.xml:an attempt to override
final parameter: dfs.support.append;  Ignoring.
Unknown argument ignored: DATA_BLOCK_ENCODING
Updating all regions with the new schema...
0/1 regions updated.
1/1 regions updated.
Done.
0 row(s) in 4.2120 seconds
{code}
4. scanned the table one more time - observed cache eviction increase

> Port HBASE-10270 'Remove DataBlockEncoding from BlockCacheKey' to trunk
> -----------------------------------------------------------------------
>
>                 Key: HBASE-10752
>                 URL: https://issues.apache.org/jira/browse/HBASE-10752
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>            Priority: Minor
>             Fix For: 0.99.0
>
>         Attachments: 10752-v1.txt, 10752-v2.txt, 10752-v3.txt, 10752-v4.txt
>
>
> The JIRA removes the block encoding from the key and forces the caller of HFileReaderV2.readBlock()
to specify the expected BlockType as well as the expected DataBlockEncoding when these matter.
This allows for a decision on either of these at read time instead of cache time, puts responsibility
where appropriate, fixes some cache misses when using the scan preloading (which does a read
without knowing the type or encoding), allows for the BlockCacheKey to be re-used by the L2
BucketCache and sets us up for a future CompoundScannerV2 which can read both un-encoded and
encoded data blocks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message