Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <968477095.1205181046923.JavaMail.jira@brutus>
Date: Mon, 10 Mar 2008 13:30:46 -0700 (PDT)
From: "stack (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-430) Performance: Scanners and getRow
 return maps with duplicate data
In-Reply-To: <3760333.1202519167579.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12577167#action_12577167 ] 

stack commented on HBASE-430:
-----------------------------


Rather than currentMillis, should you use the HCONSTANTS.LATEST_TIMESTAMP (or whatever its called)?

{code}
Index: src/java/org/apache/hadoop/hbase/HMerge.java
===================================================================
--- src/java/org/apache/hadoop/hbase/HMerge.java    (revision 635107)
+++ src/java/org/apache/hadoop/hbase/HMerge.java    (working copy)
@@ -335,8 +335,9 @@
       root = new HRegion(rootTableDir, hlog, fs, conf,
           HRegionInfo.rootRegionInfo, null, null);
 
-      HScannerInterface rootScanner = root.getScanner(COL_REGIONINFO_ARRAY,
-          new Text(), System.currentTimeMillis(), null);
+      HScannerInterface rootScanner = 
+        root.getScanner(COL_REGIONINFO_ARRAY, new Text(), 
+        System.currentTimeMillis(), null);
{code}

Made other comments up on IRC about removing commented out TODO and that there is a Writables.getHRegionInfo if you want to use instead of other Writables methods... but otherwise, patch is great removing a bunch of duplicated Map making every time we get a row

> Performance: Scanners and getRow return maps with duplicate data
> ----------------------------------------------------------------
>
>                 Key: HBASE-430
>                 URL: https://issues.apache.org/jira/browse/HBASE-430
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Bryan Duxbury
>            Assignee: Bryan Duxbury
>            Priority: Minor
>         Attachments: 430.patch
>
>
> Right now, whenever we get back multiple cells worth of data at a time, we do so in a map of HStoreKey->byte[]. This means that there is a duplicated Text row and long timestamp at the very least between every cell. This is quite a bit wasted. It also means we have to do a lot of translation every time. 
> We could create a new Writable that contains just one row, one timestamp, and a map of Text->byte[].

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.