accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3022) Limit size of data objects in ZooKeeper
Date Tue, 29 Jul 2014 14:16:38 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14077744#comment-14077744
] 

Eric Newton commented on ACCUMULO-3022:
---------------------------------------

There's a limit on the number of zookeeper watches (see ACCUMULO-2757), and watches grow with
the size of the cluster * the number tables * the number of configuration items per table.

When the number of watchers set by clients grows to 100M or so on today's hardware (10's of
Gs of ram), we should begin to worry.  I would argue that on large clusters (>1000), that
limits the number of tables to ~500-1000, which seems reasonable.

The size of a table name is not a big problem... so long as it stays under a megabyte or so.
 Sanity checking is useful to prevent user errors from tanking a whole zookeeper quorum.


> Limit size of data objects in ZooKeeper
> ---------------------------------------
>
>                 Key: ACCUMULO-3022
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3022
>             Project: Accumulo
>          Issue Type: Improvement
>    Affects Versions: 1.4.0
>            Reporter: Keith Turner
>            Priority: Minor
>             Fix For: 1.7.0
>
>
> This issue is inspired by ACCUMULO-3021.  The original issue was concerning table names,
but I've expanded the issue to include other data objects in ZooKeeper (see comments below),
especially in light of the fact that there's already an issue specifically addressing giant
table names (ACCUMULO-2366).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message