hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9857) Blockcache prefetch option
Date Tue, 20 May 2014 20:47:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14003952#comment-14003952
] 

Hadoop QA commented on HBASE-9857:
----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12645814/HBASE-9857-trunk.patch
  against trunk revision .
  ATTACHMENT ID: 12645814

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 27 new or modified
tests.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 warning messages.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 1 new Findbugs (version
1.3.9) warnings.

    {color:red}-1 release audit{color}.  The applied patch generated 1 release audit warnings
(more than the trunk's current 0 warnings).

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +  public HFileReaderV3(final Path path, FixedFileTrailer trailer, final FSDataInputStreamWrapper
fsdis,
+      family.setPrefetchBlocksOnOpen(JBoolean.valueOf(arg.delete(org.apache.hadoop.hbase.HColumnDescriptor::PREFETCH_BLOCKS_ON_OPEN)))
if arg.include?(org.apache.hadoop.hbase.HColumnDescriptor::PREFETCH_BLOCKS_ON_OPEN)

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.client.TestHCM

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9556//console

This message is automatically generated.

> Blockcache prefetch option
> --------------------------
>
>                 Key: HBASE-9857
>                 URL: https://issues.apache.org/jira/browse/HBASE-9857
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Minor
>             Fix For: 0.99.0, 0.98.3
>
>         Attachments: 9857.patch, 9857.patch, HBASE-9857-0.98.patch, HBASE-9857-trunk.patch,
HBASE-9857-trunk.patch
>
>
> Attached patch implements a prefetching function for HFile (v3) blocks, if indicated
by a column family or regionserver property. The purpose of this change is to as rapidly after
region open as reasonable warm the blockcache with all the data and index blocks of (presumably
also in-memory) table data, without counting those block loads as cache misses. Great for
fast reads and keeping the cache hit ratio high. Can tune the IO impact versus time until
all data blocks are in cache. Works a bit like CompactSplitThread. Makes some effort not to
stampede.
> I have been using this for setting up various experiments and thought I'd polish it up
a bit and throw it out there. If the data to be preloaded will not fit in blockcache, or if
as a percentage of blockcache it is large, this is not a good idea, will just blow out the
cache and trigger a lot of useless GC activity. Might be useful as an expert tuning option
though. Or not.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message