zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: ZooKeeper Limitation
Date Thu, 25 Aug 2011 19:23:32 GMT
FYI: https://issues.apache.org/jira/browse/ZOOKEEPER-1162

We should probably try to mitigate this issue, it keeps coming up.

Patrick

On Sat, Aug 13, 2011 at 12:07 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
> Yes. But getting a list of children can happen in other contexts. You should not be pushing
things to the edge like that when there are such simple ways around the problem.
>
> Sent from my iPad
>
> On Aug 13, 2011, at 1:56 AM, yosefarr <yosef.arraf@gmail.com> wrote:
>
>> So I can have one node with 500000 sub nodes(Childs) and the get Childs will fail
but get one child (getChild("/mainNode/child50")) will not fail because the request is less
than 1M is that right?
>>
>> Sent from my iPhone
>>
>> On 12 באוג 2011, at 00:36, "Ted Dunning [via zookeeper-user]"<ml-node+6678230-538638784-343064@n2.nabble.com>
wrote:
>>
>>> Same answer as before.
>>>
>>> Your request and your response have to fit into 1M byte.  How many nodes
>>> that will be depends on their names.
>>>
>>> Hint:  don't push the limit.  Design in a way that is nowhere close to the
>>> limit.
>>>
>>> On Thu, Aug 11, 2011 at 2:16 PM, yosefarr <[hidden email]> wrote:
>>>
>>>> what will work, how much maximum childs in a single node
>>>>
>>>> On Fri, Aug 12, 2011 at 12:07 AM, Ted Dunning [via zookeeper-user] <
>>>> [hidden email]> wrote:
>>>>
>>>>> 1M byte is the limit.
>>>>>
>>>>> You need to put multiple levels of directory in there.  500,000 znodes
in
>>>> a
>>>>>
>>>>> single directory won't work.
>>>>>
>>>>> On Thu, Aug 11, 2011 at 1:33 PM, yosefarr <[hidden email]<
>>>> http://user/SendEmail.jtp?type=node&node=6678116&i=0>>
>>>>> wrote:
>>>>>
>>>>>> Thanks you for your replay,
>>>>>> What do you mean that there is limit on data calls to ZK?what is
the
>>>>> limit?
>>>>>> We will have  too many direct childrens for a znode:
>>>>>> /znode
>>>>>>     /child-1
>>>>>>     /child-2
>>>>>>     ........
>>>>>>     /child-n
>>>>>> when "n" can be 500000 or more
>>>>>> in this case if i run getChildren("/znode") ,does this call will
fail?
>>>>>> or getData("/znode/child-n") will fail?
>>>>>>
>>>>>> Thanks
>>>>>> Yosef.
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 11, 2011 at 8:32 PM, Mahadev Konar-4 [via zookeeper-user]
<
>>>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=6678116&i=1>>
>>>>> wrote:
>>>>>>
>>>>>>> Yosef,
>>>>>>> There isnt a limit on the number of ZK node numbers directly.
We do
>>>>>>> have a limit on data calls to ZK. So in your case if you have
too
>>>> many
>>>>>>> direct children of a single znode (say millions), the getChildren
>>>>>>> calls return an error. I'd advise against having too many direct
>>>>>>> chidlren for a znode. Bucketing the users might make sense for
you if
>>>>>>> you have too many users.
>>>>>>>
>>>>>>> thanks
>>>>>>> mahadev
>>>>>>>
>>>>>>> On Thu, Aug 11, 2011 at 2:19 AM, yosefarr <[hidden email]<
>>>>>> http://user/SendEmail.jtp?type=node&node=6677260&i=0>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> If i want to save users that connect to my system as zookeeper
>>>> nodes
>>>>>> and
>>>>>>>> each user has some proprties - connection id, user name and
etc:
>>>>>>>> i need to know if there is a Limitation on ZooKeeper nodes
numbers?
>>>>>>>>
>>>>>>>> /server1
>>>>>>>>     /user-1
>>>>>>>>     /user-2
>>>>>>>>     ........
>>>>>>>>     /user-n
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Yosef.
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>
>>>>>>
>>>>>
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6675643.html
>>>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> If you reply to this email, your message will be added to the
>>>>> discussion
>>>>>>> below:
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677260.html
>>>>>>> To unsubscribe from ZooKeeper Limitation, click here<
>>>>>>
>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>>
>>>>>
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6677984.html
>>>>>
>>>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> If you reply to this email, your message will be added to the discussion
>>>>> below:
>>>>>
>>>>>
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678116.html
>>>>> To unsubscribe from ZooKeeper Limitation, click here<
>>>>> .
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678146.html
>>>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>>>>
>>>
>>>
>>> If you reply to this email, your message will be added to the discussion below:
>>> http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6678230.html
>>> To unsubscribe from ZooKeeper Limitation, click here.
>>
>>
>> --
>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/ZooKeeper-Limitation-tp6675643p6682887.html
>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>

Mime
View raw message