accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ACCUMULO-551) Experiment with multi-node batch writer
Date Fri, 20 Apr 2012 22:43:33 GMT
Keith Turner created ACCUMULO-551:
-------------------------------------

             Summary: Experiment with multi-node batch writer
                 Key: ACCUMULO-551
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-551
             Project: Accumulo
          Issue Type: Task
            Reporter: Keith Turner
             Fix For: 1.5.0


Accumulo has a batch writer that batches mutations by tablet server for writes.  This works
well until there are alot of tablet servers being written to at which point only a small amount
of data is being sent to each tablet server.  Would it be better for the client to batch writes
for multiple tablet servers and send them to one server which writes directly to the tablet
servers?  

One possible way to do this is to :
 
 * batch mutations by rack on the client
 * send all of those mutations to one random tablet server on the rack 
 * have the random tablet server write to the other servers on the rack

This cuts down on the number of direct connections the client has to make.  Could have the
following benefits.

 * Tablet servers can keep connections open to other tablet servers.
 * A write pipeline

Would be interesting to run some test and see how well this works.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message