accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-1754) support scale-up behavior in BatchWriter
Date Mon, 07 Oct 2013 20:30:41 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788527#comment-13788527
] 

Keith Turner commented on ACCUMULO-1754:
----------------------------------------

This issue complements ACCUMULO-1083. 

TableServerBatchReader does something like this.  If it has more threads than tablet servers,
then it start splitting things up by tablet and sending concurrent request to tablet servers.

> support scale-up behavior in BatchWriter
> ----------------------------------------
>
>                 Key: ACCUMULO-1754
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1754
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: client
>            Reporter: Adam Fuchs
>
> The o.a.a.client.impl.TabletServerBatchWriter currently limits write threads to one per
server. Some people want to use Accumulo on beefy servers where sending one batch at a time
is a bottleneck. We can improve the performance on small clusters of beefy nodes by splitting
up write batches by tablet and server.
> This should be done in such a way that we can still batch across tablets when the cluster
is large. Basically, mutations should be binned by server, then further binned by tablet up
to the limit of (some factor of) threads in the BatchWriter's thread pool.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message