hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cosmin Lehene (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2066) Perf: parallelize puts
Date Tue, 09 Feb 2010 19:23:28 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831603#action_12831603
] 

Cosmin Lehene commented on HBASE-2066:
--------------------------------------

Patch fails to apply on trunk.
After manually applying chunks I got these while doing puts

EXCEPTION 1

java.lang.NullPointerException
  at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.deleteCachedLocation(HConnectionManager.java:889)
  at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfPuts(HConnectionManager.java:1413)
  at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:586)
  at org.apache.hadoop.hbase.client.HTable.put(HTable.java:471)
  at TestBatchPut$MyThread.run(TestBatchPut.java:65)


EXCEPTION 2

java.lang.NullPointerException
  at java.util.TreeMap.rotateRight(TreeMap.java:2057)
  at java.util.TreeMap.fixAfterDeletion(TreeMap.java:2217)
  at java.util.TreeMap.deleteEntry(TreeMap.java:2151)
  at java.util.TreeMap.remove(TreeMap.java:585)
  at org.apache.hadoop.hbase.util.SoftValueSortedMap.remove(SoftValueSortedMap.java:104)
  at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.deleteCachedLocation(HConnectionManager.java:897)
  at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfPuts(HConnectionManager.java:1413)
  at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:586)
  at org.apache.hadoop.hbase.client.HTable.put(HTable.java:471)
  at TestBatchPut$MyThread.run(TestBatchPut.java:65)


Also the throughput went down and the max seconds for a put went up (could be also from the
hbase restart).

I'll attach the piece of code I'm using to benchmark it

> Perf: parallelize puts
> ----------------------
>
>                 Key: HBASE-2066
>                 URL: https://issues.apache.org/jira/browse/HBASE-2066
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>            Reporter: ryan rawson
>            Assignee: ryan rawson
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2066-branch.patch, HBASE-2066-v2.patch
>
>
> Right now with large region count tables, the write buffer is not efficient.  This is
because we issue potentially N RPCs, where N is the # of regions in the table.  When N gets
large (lets say 1200+) things become sloowwwww.
> Instead if we batch things up using a different RPC and use thread pools, we could see
higher performance!
> This requires a RPC change...

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message