Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
Message-ID: <5759C7DA.5000303@gmail.com>
Date: Thu, 09 Jun 2016 15:47:38 -0400
From: Josh Elser <josh.elser@gmail.com>
User-Agent: Postbox 3.0.11 (Macintosh/20140602)
MIME-Version: 1.0
To: user@accumulo.apache.org
Subject: Re: visibility constraint performance
References: <CAKERyGSOdm7d+S20rkN9-vkZqGGt-8W1ws=PQS2Rcx5GqDVyvw@mail.gmail.com> <CAGUtCHrF7tj0Cr+WdGsFnDC220BgpdS5P6aPK4FWkLTt7jixOA@mail.gmail.com>
In-Reply-To: <CAGUtCHrF7tj0Cr+WdGsFnDC220BgpdS5P6aPK4FWkLTt7jixOA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
archived-at: Thu, 09 Jun 2016 19:47:47 -0000


Keith Turner wrote:
>
>
> On Thu, Jun 9, 2016 at 2:49 PM, Jonathan Wonders <jwonders88@gmail.com
> <mailto:jwonders88@gmail.com>> wrote:
>
>     Hi All,
>
>     I've been tracking down some performance issues on a few 1.6.x
>     environments and noticed some interesting and potentially
>     undesirable behavior in the visibility constraint.  When the
>     associated visibility evaluator checks a token to see if the user
>     has a matching authorization, it uses the security operations from
>     the tablet server's constraint environment which ends up
>     authenticating the user's credentials on each call.  This will end
>     up flooding logs with Audit messages corresponding to these
>     authentications if the Audit logging is enabled.  It also consumes a
>     non-negligible amount of CPU, produces a lot of garbage (maybe
>     50-60% of that generated under a heavy streaming ingest load), and
>     can cause some contention between client pool threads when accessing
>     the ZooCache.
>
>     My initial measurements indicate a 25-30% decrease in ingest rate
>     (entries/s and MB/s) for my environement and workload when this
>     constraint is enabled.  This is with the Audit logging disabled.
>
>     Is this intended behavior?  It seems like the authentication is
>     redundant with the authentication that is performed at the beginning
>     of the update session.
>
>
> No. It would be best to avoid that behavior.

Agreed. Want to open up something on JIRA? It sounds like there might be 
a few things we can investigate.

* Synchronization/concurrency on ZooCache
* Excessive object creation when using the VisibilityConstraint
* Noticeable time spent creating Audit messages which are not logged 
(Auditing is disabled)

I miss any points?