accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-4039) try out a proactor design pattern for tserver services
Date Tue, 27 Oct 2015 19:38:27 GMT


Eric Newton commented on ACCUMULO-4039:

Could you be a little more specific?

The current RPC model is Half-Sync, Half-Async (HSHA). So requests and responses are handled
asynchronously, but the RPC itself is executed by a shared thread-pool.

This is a convenient paradigm to program against. Some things are tricky (like the service
blocking on itself for meta services), but most requests are implemented as straight-line
code: no callbacks, Futures and other bookkeeping.

What limits are you pushing up against, and how would a different design solve it?

> try out a proactor design pattern for tserver services
> ------------------------------------------------------
>                 Key: ACCUMULO-4039
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>            Reporter: Adam Fuchs
>            Priority: Minor
> For large instances (i.e. lots of clients for a given tserver) we create oodles of threads
on the tserver. This makes for difficulty in predicting performance, memory usage, etc. Moreover,
we have operations that recurse, like a server querying itself, that we currently solve by
having separate thread pools for regular table operations and metadata table operations, and
we "disallow" things like an iterator writing to another table. One alternative option would
be to switch to a Proactor pattern:
> The core of this would be to switch to using a selection set rather than a thread per
active connection, and then wrap everything in sessions that make progress in something like
a state model, with states that account for asynchronous communications and remote work.

This message was sent by Atlassian JIRA

View raw message