accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: Any limits on Accumulo namespaces/tables?
Date Thu, 25 Jun 2015 20:21:45 GMT
Correction: s/this because a big problem/this causes a big problem/ :)

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Thu, Jun 25, 2015 at 2:38 PM, Christopher <ctubbsii@apache.org> wrote:
> In theory, no, there aren't limits. In practice, you'd be limited by
> the amount of memory you have available in ZooKeeper, and you may have
> trouble (currently) with lots of ZooKeeper watchers which monitor
> configuration changes for tables.
>
> There may be data structures in the API and throughout the code which
> contain a complete list of tables, also, and that could hurt in terms
> of memory.
>
> In short, we don't define a limit, but there may be bottlenecks you
> might encounter. If you do find these, we may be able to make
> improvements, so let us know. If this because a big problem, we may
> impose some upper bound in the future, just so people can reason about
> their environment better, and we could offer more concrete numbers on
> what is feasible.
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
>
> On Thu, Jun 25, 2015 at 2:31 PM, Konstantin Pelykh <kpelykh@gmail.com> wrote:
>> Hi everyone,
>>
>> Are there any limits on a number of namespaces and tables in Accumulo?
>> For my current application I'm planning to use multiple namespaces to
>> separate data for different clients. Each namespace will have 8 tables.
>> Would it be posible to have 100,500, 1000 namespaces in Accumulo? How will
>> such namespace layout impact performance of accumulo scanner? As far as I
>> rememebr every time when scanner is created  it gets the whole list of
>> namespaces/tables from zookeeper and it slows down performace as number of
>> namespaces increases.
>>
>> Thanks!
>> Konstantin
>> --------
>> Big Data / Search Consultant
>> LinkedIn: linkedin.com/in/kpelykh
>> Website: www.kpelykh.com

Mime
View raw message