accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-1085) make the number of threads for assignment configurable
Date Tue, 12 Mar 2013 19:43:12 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600387#comment-13600387
] 

Keith Turner commented on ACCUMULO-1085:
----------------------------------------

bq. We could farm out RecoveryFile-to-RFile conversion to the tserver work queue. Once complete,
the log could be replaced by an RFile and loaded without further recovery.

The written rfile will not be local to the tserver in this case.

Still need to solve the problem of using too much memory (if multiple threads are servicing
queue on each tablet server).  One possible way to solve the memory problem is to write multiple
rfiles per tablet instead of just one during recovery.  Seems like everyone may be working
twoards the conclusion that the log sort itself needs to produce a different output than one
giant map file.  This would defintely be a 1.6 change and would need to be designed in concert
with a few other tickets for 1.6. 
                
> make the number of threads for assignment configurable
> ------------------------------------------------------
>
>                 Key: ACCUMULO-1085
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1085
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>            Reporter: Eric Newton
>            Assignee: Keith Turner
>             Fix For: 1.6.0
>
>         Attachments: ACCUMULO-1085.patch
>
>
> Log recoveries from sorted logs take considerable time: run them in parallel with more
assignment threads.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message