hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-430) Performance: Scanners and getRow return maps with duplicate data
Date Mon, 10 Mar 2008 20:30:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12577167#action_12577167
] 

stack commented on HBASE-430:
-----------------------------


Rather than currentMillis, should you use the HCONSTANTS.LATEST_TIMESTAMP (or whatever its
called)?

{code}
Index: src/java/org/apache/hadoop/hbase/HMerge.java
===================================================================
--- src/java/org/apache/hadoop/hbase/HMerge.java    (revision 635107)
+++ src/java/org/apache/hadoop/hbase/HMerge.java    (working copy)
@@ -335,8 +335,9 @@
       root = new HRegion(rootTableDir, hlog, fs, conf,
           HRegionInfo.rootRegionInfo, null, null);
 
-      HScannerInterface rootScanner = root.getScanner(COL_REGIONINFO_ARRAY,
-          new Text(), System.currentTimeMillis(), null);
+      HScannerInterface rootScanner = 
+        root.getScanner(COL_REGIONINFO_ARRAY, new Text(), 
+        System.currentTimeMillis(), null);
{code}

Made other comments up on IRC about removing commented out TODO and that there is a Writables.getHRegionInfo
if you want to use instead of other Writables methods... but otherwise, patch is great removing
a bunch of duplicated Map making every time we get a row

> Performance: Scanners and getRow return maps with duplicate data
> ----------------------------------------------------------------
>
>                 Key: HBASE-430
>                 URL: https://issues.apache.org/jira/browse/HBASE-430
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Bryan Duxbury
>            Assignee: Bryan Duxbury
>            Priority: Minor
>         Attachments: 430.patch
>
>
> Right now, whenever we get back multiple cells worth of data at a time, we do so in a
map of HStoreKey->byte[]. This means that there is a duplicated Text row and long timestamp
at the very least between every cell. This is quite a bit wasted. It also means we have to
do a lot of translation every time. 
> We could create a new Writable that contains just one row, one timestamp, and a map of
Text->byte[].

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message