accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Updated] (ACCUMULO-2367) Revisit thrift settings
Date Sat, 04 Apr 2015 02:59:33 GMT


Josh Elser updated ACCUMULO-2367:
    Fix Version/s:     (was: 1.7.0)

> Revisit thrift settings
> -----------------------
>                 Key: ACCUMULO-2367
>                 URL:
>             Project: Accumulo
>          Issue Type: Task
>          Components: gc, master, trace, tserver
>            Reporter: John Vines
>             Fix For: 1.8.0
> Related to ACCUMULO-2360 and ACCUMULO-2352.
> There are 2 thrift options in play that we use with the max message size (one introduced
under 2360). The new setting is used for reading chunks of data of the network. Specifically,
it gets a size, allocates a buffer, and then reads the remainder of that chunk into that buffer.
If it gets garbage data that is a positive int, it will allocate that data.
> It is then on top of that layer where it will reassemble frames (TFramedTransport). This
is where we had the old setting. Specifically, it makes sure that really large thrift calls
get rejected (with a max message size of 1g, the default, I could send a create table request
up to 1g without it being outright rejected). This frame consists of smaller chunks, so the
setting in the first paragraph isn't in play.
> What this means is that, contrary to the logic in 2360, we should have two settings.
The former, I believe, it just used to handle noise on the network without breaking things.
And the size of chunks things get written in, but I'm not quite sure as I couldn't find the
opposing code for the write. The latter is used for enforcing decent sized requests. So we
should probably have a single value for all lowest level thrift pools (MAX_BUFFER_SIZE), but
have different knobs for each service because the tserver will get the largest frames (mutations)
while the master, gc, and tracer expect substantially smaller requests.
> Also, I think the tracer is vulnerable to 2360 as well, since it seems to have no frame
size limiting.

This message was sent by Atlassian JIRA

View raw message