Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Received-SPF: pass (nike.apache.org: domain of josh.elser@gmail.com designates
 209.85.216.169 as permitted sender)
Message-ID: <50E62696.2020203@gmail.com>
Date: Thu, 03 Jan 2013 19:47:18 -0500
From: Josh Elser <josh.elser@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:10.0.11) Gecko/20121202 Thunderbird/10.0.11
MIME-Version: 1.0
To: dev@accumulo.apache.org
Subject: Re: ingest performance oscillations and Xceivers
References: <FB59676C-97D4-4447-A5CD-829B82F8892E@ll.mit.edu>
 <CADxc9BnhBwHHjT183GgKDj-GW-gsrTW6URm5qcEmgKM6txbT2A@mail.gmail.com>
 <20130102191131.GA12538@ll.mit.edu>
 <CAGUtCHrt6q-Mtshs41C7VEW0uHxGwpVDbtd0OKGQp=sq3SP3hw@mail.gmail.com>
 <7C21BEED-BDBD-40D4-97CF-EF60B776540A@ll.mit.edu>
 <CAMz+DutJkNdGLYjXhvJj=j9ifm7t85MPb_hW73wwh022E1v5fw@mail.gmail.com>
 <20130103222534.GA23212@ll.mit.edu>
 <CADczPYQRVhZwjJO1sWeZvNYZpi7GFEq23ZRFesb_4+NsyWBBYA@mail.gmail.com>
 <40FDB156-E32B-41C0-B483-EA69B2178586@ll.mit.edu>
In-Reply-To: <40FDB156-E32B-41C0-B483-EA69B2178586@ll.mit.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I think you missed your attachment :)

On 01/03/2013 07:31 PM, Kepner, Jeremy - 0553 - MITLL wrote:
> Attached is the log for a set for 3 ingests.  The first hump is with 1 ingestor, the second hump is with 2 ingestors, and the third hump is with 3 ingestors.  The 3 ingestor case starts oscillating about 21:40. I don't see any spikes in any of the fields.  It would be nice if there was also a plot of Xceivers since I think that anti-correlates.  I do sometimes see the hold times highlighted in red.  The values are usually in hundreds of msec.
>
> On Jan 3, 2013, at 6:04 PM, John Vines wrote:
>
>> How many hard drives and what's your max minor/maor compactions set to?
>> These can severely limit your performance due to potential disk thrashing.
>> If you observe the monitor, when ingest starts trailing, do you see it
>> undergoing, or possibly being backed up on, any form of compaction? And
>> lastly, are you see the bug eric mentioned above? See if the monitor is
>> reporting any warnings like those mentioned.