hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-9775) Client write path perf issues
Date Wed, 30 Oct 2013 05:18:29 GMT

     [ https://issues.apache.org/jira/browse/HBASE-9775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-9775:
-------------------------

    Attachment: 9775.rig.txt

I'm trying to write a rig that the client can run in so we can inspect it.  Attached is a
bit of code that mocks a cluster of 1k servers and 100k regions.  Currently it runs w/o throwing
exceptions of failures.

When I put it under the profiler, we spin up 7 threads and that seems to keep us running nicely;
we never go beyond 7.

If I add some friction by adding pause to the mock Put handler so it "takes time" to process
the puts, thread count spins up and tops out at 100 which looks like it is: AsyncProcess#maxTotalConcurrentTasks
whose config is hbase.client.max.total.tasks.

I suppose I should randomize up the way I put -- it is sort of ordered at the moment but even
then, it looks like I'd be doing 1/10th of the servers at a time.

Let me update and see what the [~liochon] recent changes do.


> Client write path perf issues
> -----------------------------
>
>                 Key: HBASE-9775
>                 URL: https://issues.apache.org/jira/browse/HBASE-9775
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.96.0
>            Reporter: Elliott Clark
>            Priority: Critical
>         Attachments: 9775.rig.txt, Charts Search   Cloudera Manager - ITBLL.png, Charts
Search   Cloudera Manager.png, hbase-9775.patch, job_run.log, short_ycsb.png, ycsb_insert_94_vs_96.png,
ycsb.png
>
>
> Testing on larger clusters has not had the desired throughput increases.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message