hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-430) Performance: Scanners and getRow return maps with duplicate data
Date Wed, 12 Mar 2008 04:10:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12577695#action_12577695
] 

stack commented on HBASE-430:
-----------------------------

Last patch builds successfully on my machine.  Before committing, you might make the following
changes:

In src/java/org/apache/hadoop/hbase/client/HBaseAdmin.java remove:

+          // HStoreKey key = (HStoreKey) e.getKey();

In src/java/org/apache/hadoop/hbase/client/HConnectionManager.java remove:

+            // for (Map.Entry<Text, Cell> e: values.entrySet()) {
+            //   if (e.getKey().equals(COL_REGIONINFO)) {
+            //     // HRegionInfo info = new HRegionInfo();
+            //     // info = (HRegionInfo) Writables.getWritable(
+            //     //   e.getValue().getValue(), info);
+            // 
+            //   }
+            // }

In src/java/org/apache/hadoop/hbase/io/RowResult.java, copyright should be 2008, not 2007.
 Thats kinda sweet that you have it implement Map

Whats going on here?  + 
+  public Set<Text> keySet() {
+    Set<Text> result = new HashSet<Text>();
+    for (Writable w : cells.keySet()) {
+      result.add((Text)w);
+    }
+    return result;
+  }

You are trying to protect against client alterations of underlying cell?  If so, instead wrap
in a call to Collections.unmodifiableSet: http://java.sun.com/j2se/1.5.0/docs/api/java/util/Collections.html#unmodifiableSet(java.util.Set)

There are a few other places where you do similar copies.

+1 on patch after consideration of above.





> Performance: Scanners and getRow return maps with duplicate data
> ----------------------------------------------------------------
>
>                 Key: HBASE-430
>                 URL: https://issues.apache.org/jira/browse/HBASE-430
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Bryan Duxbury
>            Assignee: Bryan Duxbury
>            Priority: Minor
>         Attachments: 430-v2.patch, 430-v3.patch, 430-v4.patch, 430.patch
>
>
> Right now, whenever we get back multiple cells worth of data at a time, we do so in a
map of HStoreKey->byte[]. This means that there is a duplicated Text row and long timestamp
at the very least between every cell. This is quite a bit wasted. It also means we have to
do a lot of translation every time. 
> We could create a new Writable that contains just one row, one timestamp, and a map of
Text->byte[].

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message