accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: visibility constraint performance
Date Thu, 09 Jun 2016 19:47:38 GMT


Keith Turner wrote:
>
>
> On Thu, Jun 9, 2016 at 2:49 PM, Jonathan Wonders <jwonders88@gmail.com
> <mailto:jwonders88@gmail.com>> wrote:
>
>     Hi All,
>
>     I've been tracking down some performance issues on a few 1.6.x
>     environments and noticed some interesting and potentially
>     undesirable behavior in the visibility constraint.  When the
>     associated visibility evaluator checks a token to see if the user
>     has a matching authorization, it uses the security operations from
>     the tablet server's constraint environment which ends up
>     authenticating the user's credentials on each call.  This will end
>     up flooding logs with Audit messages corresponding to these
>     authentications if the Audit logging is enabled.  It also consumes a
>     non-negligible amount of CPU, produces a lot of garbage (maybe
>     50-60% of that generated under a heavy streaming ingest load), and
>     can cause some contention between client pool threads when accessing
>     the ZooCache.
>
>     My initial measurements indicate a 25-30% decrease in ingest rate
>     (entries/s and MB/s) for my environement and workload when this
>     constraint is enabled.  This is with the Audit logging disabled.
>
>     Is this intended behavior?  It seems like the authentication is
>     redundant with the authentication that is performed at the beginning
>     of the update session.
>
>
> No. It would be best to avoid that behavior.

Agreed. Want to open up something on JIRA? It sounds like there might be 
a few things we can investigate.

* Synchronization/concurrency on ZooCache
* Excessive object creation when using the VisibilityConstraint
* Noticeable time spent creating Audit messages which are not logged 
(Auditing is disabled)

I miss any points?

Mime
View raw message