accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Vines (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ACCUMULO-2367) Revisit thrift settings
Date Thu, 13 Feb 2014 20:57:21 GMT
John Vines created ACCUMULO-2367:
------------------------------------

             Summary: Revisit thrift settings
                 Key: ACCUMULO-2367
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2367
             Project: Accumulo
          Issue Type: Task
          Components: gc, master, trace, tserver
            Reporter: John Vines
             Fix For: 1.7.0


Related to ACCUMULO-2360 and ACCUMULO-2352.

There are 2 thrift options in play that we use with the max message size (one introduced under
2360). The new setting is used for reading chunks of data of the network. Specifically, it
gets a size, allocates a buffer, and then reads the remainder of that chunk into that buffer.
If it gets garbage data that is a positive int, it will allocate that data.

It is then on top of that layer where it will reassemble frames (TFramedTransport). This is
where we had the old setting. Specifically, it makes sure that really large thrift calls get
rejected (with a max message size of 1g, the default, I could send a create table request
up to 1g without it being outright rejected). This frame consists of smaller chunks, so the
setting in the first paragraph isn't in play.

What this means is that, contrary to the logic in 2360, we should have two settings. The former,
I believe, it just used to handle noise on the network without breaking things. And the size
of chunks things get written in, but I'm not quite sure as I couldn't find the opposing code
for the write. The latter is used for enforcing decent sized requests. So we should probably
have a single value for all lowest level thrift pools (MAX_BUFFER_SIZE), but have different
knobs for each service because the tserver will get the largest frames (mutations) while the
master, gc, and tracer expect substantially smaller requests.

Also, I think the tracer is vulnerable to 2360 as well, since it seems to have no frame size
limiting.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message