Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Thu, 13 Feb 2014 20:57:21 +0000 (UTC)
From: "John Vines (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12695074.1392325019431.38753.1392325041412@arcas>
In-Reply-To: <JIRA.12695074.1392325019431@arcas>
References: <JIRA.12695074.1392325019431@arcas>
Subject: [jira] [Created] (ACCUMULO-2367) Revisit thrift settings
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

John Vines created ACCUMULO-2367:
------------------------------------

             Summary: Revisit thrift settings
                 Key: ACCUMULO-2367
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2367
             Project: Accumulo
          Issue Type: Task
          Components: gc, master, trace, tserver
            Reporter: John Vines
             Fix For: 1.7.0


Related to ACCUMULO-2360 and ACCUMULO-2352.

There are 2 thrift options in play that we use with the max message size (one introduced under 2360). The new setting is used for reading chunks of data of the network. Specifically, it gets a size, allocates a buffer, and then reads the remainder of that chunk into that buffer. If it gets garbage data that is a positive int, it will allocate that data.

It is then on top of that layer where it will reassemble frames (TFramedTransport). This is where we had the old setting. Specifically, it makes sure that really large thrift calls get rejected (with a max message size of 1g, the default, I could send a create table request up to 1g without it being outright rejected). This frame consists of smaller chunks, so the setting in the first paragraph isn't in play.

What this means is that, contrary to the logic in 2360, we should have two settings. The former, I believe, it just used to handle noise on the network without breaking things. And the size of chunks things get written in, but I'm not quite sure as I couldn't find the opposing code for the write. The latter is used for enforcing decent sized requests. So we should probably have a single value for all lowest level thrift pools (MAX_BUFFER_SIZE), but have different knobs for each service because the tserver will get the largest frames (mutations) while the master, gc, and tracer expect substantially smaller requests.

Also, I think the tracer is vulnerable to 2360 as well, since it seems to have no frame size limiting.


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)