From dev-return-717-archive-asf-public=cust-asf.ponee.io@fluo.apache.org Wed Jan 10 17:54:04 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 9E15D18072F for ; Wed, 10 Jan 2018 17:54:04 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 8E436160C2E; Wed, 10 Jan 2018 16:54:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id DF0A4160C23 for ; Wed, 10 Jan 2018 17:54:03 +0100 (CET) Received: (qmail 79292 invoked by uid 500); 10 Jan 2018 16:54:03 -0000 Mailing-List: contact dev-help@fluo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@fluo.apache.org Delivered-To: mailing list dev@fluo.apache.org Received: (qmail 79280 invoked by uid 99); 10 Jan 2018 16:54:03 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Jan 2018 16:54:03 +0000 Received: from mail-it0-f54.google.com (mail-it0-f54.google.com [209.85.214.54]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id C421B1A008B for ; Wed, 10 Jan 2018 16:53:59 +0000 (UTC) Received: by mail-it0-f54.google.com with SMTP id u62so156821ita.2 for ; Wed, 10 Jan 2018 08:53:58 -0800 (PST) X-Gm-Message-State: AKwxytcg4/q6MV8t+zHMXtlHBbymMRyKo+60tc1qorSIdT2EmJtVmLA0 GsVRy4y+WEwCbVWFhVPIeuTcOAfTSOGvwL8chD4= X-Google-Smtp-Source: ACJfBouO6HVZ5mI2oBIyF07zv5hIIEU91dyS6vhNklNvZfinfiyxOm2Iku4VAm1pbemIvvKpOqgVV2R2SDHicUmok/c= X-Received: by 10.36.123.84 with SMTP id q81mr5417938itc.84.1515603238137; Wed, 10 Jan 2018 08:53:58 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.158.220 with HTTP; Wed, 10 Jan 2018 08:53:17 -0800 (PST) From: Mike Walch Date: Wed, 10 Jan 2018 11:53:17 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Successful Stress Test Run To: dev@fluo.apache.org Content-Type: multipart/alternative; boundary="001a1146f0eca1353f05626ee0fc" --001a1146f0eca1353f05626ee0fc Content-Type: text/plain; charset="UTF-8" Nice work. Is there anyway to compare this release with previous ones? On Wed, Jan 10, 2018 at 10:19 AM, Keith Turner wrote: > I completed a successful 24hr run of the Fluo stress test on a 10 node > EC2 cluster. For the test 1 billion random integers were loaded via > map reduce and then 370 million were loaded by Fluo. This resulted in > ~1.3 billion transaction executing and ~13 million collisions. Fluo > commit dbad51d was used for the test. Below is the final output from > the test. > > *****Verifying Fluo & MapReduce results match***** > Success! Fluo & MapReduce both calculated 1369064132 unique integers > > During the test CPU utilization was not uniformly high. Looking at > the Accumulo monitor some nodes would have lots of queued scans. > Running jstack on that nodes showed lots of threads trying to reserve > open files. However there were only a few threads actually running > scans. This seemed very odd and I plan to investigate further. I had > set the max open files to 1000 and all tablets had only 3 to 4 files. > Therefore if 1000 files were reserved I would have expected to see > lots of scans running, however this was not what I saw. > > > Below is a gist with info about config used for the test. > > https://gist.github.com/keith-turner/e28ee6cd4941210f34e5cd0e6a6b3106 > --001a1146f0eca1353f05626ee0fc--