accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: Any limits on Accumulo namespaces/tables?
Date Thu, 25 Jun 2015 18:38:56 GMT
In theory, no, there aren't limits. In practice, you'd be limited by
the amount of memory you have available in ZooKeeper, and you may have
trouble (currently) with lots of ZooKeeper watchers which monitor
configuration changes for tables.

There may be data structures in the API and throughout the code which
contain a complete list of tables, also, and that could hurt in terms
of memory.

In short, we don't define a limit, but there may be bottlenecks you
might encounter. If you do find these, we may be able to make
improvements, so let us know. If this because a big problem, we may
impose some upper bound in the future, just so people can reason about
their environment better, and we could offer more concrete numbers on
what is feasible.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Thu, Jun 25, 2015 at 2:31 PM, Konstantin Pelykh <kpelykh@gmail.com> wrote:
> Hi everyone,
>
> Are there any limits on a number of namespaces and tables in Accumulo?
> For my current application I'm planning to use multiple namespaces to
> separate data for different clients. Each namespace will have 8 tables.
> Would it be posible to have 100,500, 1000 namespaces in Accumulo? How will
> such namespace layout impact performance of accumulo scanner? As far as I
> rememebr every time when scanner is created  it gets the whole list of
> namespaces/tables from zookeeper and it slows down performace as number of
> namespaces increases.
>
> Thanks!
> Konstantin
> --------
> Big Data / Search Consultant
> LinkedIn: linkedin.com/in/kpelykh
> Website: www.kpelykh.com

Mime
View raw message