hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2066) Perf: parallelize puts
Date Sun, 21 Mar 2010 11:07:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847913#action_12847913
] 

Andrew Purtell commented on HBASE-2066:
---------------------------------------

Since HBASE-2066 was committed on 0.20 branch o.a.h.h.client.TestGetRowVersions is hanging.


Before:

{noformat}
test-core:
    [mkdir] Created dir: /home/apurtell/src/Hadoop/hbase.git/build/test/logs
    [junit] Running org.apache.hadoop.hbase.client.TestGetRowVersions
    [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 36.048 sec
{noformat}

Now:

{noformat}
test-core:
    [mkdir] Created dir: /home/apurtell/src/Hadoop/hbase.git/build/test/logs
    [junit] Running org.apache.hadoop.hbase.client.TestGetRowVersions
    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
    [junit] Test org.apache.hadoop.hbase.client.TestGetRowVersions FAILED (timeout)
{noformat}

TestGetRowVersions shuts down and restarts the minicluster mid test. Maybe it could just force
flush instead? 

Prior to 2066 this test would exit, but I think only by luck. Now, according to jstack main()
is joined to the regionserver thread, which is trying again and again to report for duty to
a master thread that has gone away. Neither main nor the regionserver threads are daemon threads,
so the JVM does not exit.

> Perf: parallelize puts
> ----------------------
>
>                 Key: HBASE-2066
>                 URL: https://issues.apache.org/jira/browse/HBASE-2066
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.2, 0.20.3
>            Reporter: ryan rawson
>            Assignee: ryan rawson
>             Fix For: 0.20.4, 0.21.0
>
>         Attachments: HBASE-2066-20-branch.txt, HBASE-2066-3.patch, HBASE-2066-branch.patch,
HBASE-2066-v2.patch, TestBatchPut.java
>
>
> Right now with large region count tables, the write buffer is not efficient.  This is
because we issue potentially N RPCs, where N is the # of regions in the table.  When N gets
large (lets say 1200+) things become sloowwwww.
> Instead if we batch things up using a different RPC and use thread pools, we could see
higher performance!
> This requires a RPC change...

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message