Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <1871476146.1246641467247.JavaMail.jira@brutus>
Date: Fri, 3 Jul 2009 10:17:47 -0700 (PDT)
From: "stack (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-1597) Prevent unnecessary caching of
 blocks during compactions
In-Reply-To: <1146530730.1246424567169.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12727054#action_12727054 ] 

stack commented on HBASE-1597:
------------------------------

This seems odd:

{code}
+    public CompactionReader(Reader reader) {
+      super(reader.istream, reader.fileSize, reader.cache);
+      super.blockIndex = reader.blockIndex;
+      super.trailer = reader.trailer;
+      super.lastkey = reader.lastkey;
+      super.avgKeyLen = reader.avgKeyLen;
+      super.avgValueLen = reader.avgValueLen;
+      super.comparator = reader.comparator;
+      super.metaIndex = reader.metaIndex;
+      super.fileInfoLoaded = reader.fileInfoLoaded;
+      super.compressAlgo = reader.compressAlgo;
+    }
{code}

Why not make a constructor that takes above args?

Too much of readBlock is duplicated in CompactionReader.  Would suggest changing backing readBlock so it makes call to a new method that does the add-to-cache... then subclass this new method only.

Otherwise, nice patch.

> Prevent unnecessary caching of blocks during compactions
> --------------------------------------------------------
>
>                 Key: HBASE-1597
>                 URL: https://issues.apache.org/jira/browse/HBASE-1597
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.0
>            Reporter: Jonathan Gray
>            Assignee: Jonathan Gray
>             Fix For: 0.20.0
>
>         Attachments: HBASE-1597-v1.patch
>
>
> When running any kind of compaction, we read every block of every storefile being compacted into the block cache.
> We would like to reuse any already cached blocks, if available, but otherwise we should not bog down the LRU with unnecessary blocks.
> This is not as bad as it was with the old LRU because the latest LRU implementation (HBASE-1460) is scan-resistant.  This ensures that we are not causing massive eviction of the blocks that are being read multiple times or from in-memory tables.  However, this does add to the GC-woes of an import because each block gets further referenced, and for longer periods of time.  There is also overhead in running the LRU evictions.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.