accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Bella (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4039) try out a proactor design pattern for tserver services
Date Tue, 27 Oct 2015 20:06:27 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977060#comment-14977060
] 

Ivan Bella commented on ACCUMULO-4039:
--------------------------------------

The Proactor pattern certainly seems an interesting approach having read the paper.  What
I am missing is how this approach will make predicting performance or memory usage easier.
 Currently we know how many threads in each pool there are, and how much memory we can use
in each thread, and how many processors we have.  This model does not appear to be difficult
to predict but I am sure I am missing something.

On the positive side this may enable better reuse of threads for "operations that recurse."
 What I do not want to happen however is allowing one set of scans to somehow block scans
against the metadata table.  The current approach of having the separate thread pools makes
this easy to configure and control.  QoS needs be be considered.


> try out a proactor design pattern for tserver services
> ------------------------------------------------------
>
>                 Key: ACCUMULO-4039
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4039
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>            Reporter: Adam Fuchs
>            Priority: Minor
>
> For large instances (i.e. lots of clients for a given tserver) we create oodles of threads
on the tserver. This makes for difficulty in predicting performance, memory usage, etc. Moreover,
we have operations that recurse, like a server querying itself, that we currently solve by
having separate thread pools for regular table operations and metadata table operations, and
we "disallow" things like an iterator writing to another table. One alternative option would
be to switch to a Proactor pattern: https://en.wikipedia.org/wiki/Proactor_pattern
> The core of this would be to switch to using a selection set rather than a thread per
active connection, and then wrap everything in sessions that make progress in something like
a state model, with states that account for asynchronous communications and remote work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message