curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Chen <cc...@twitter.com>
Subject Re: Best Practice while creating znodes?
Date Mon, 14 Sep 2015 23:56:12 GMT
Tread carefully!

Yes it's configurable but it only takes one client that barfs on a large
payload to ruin your day (or at least make it really weird).

Jordan is right, it applies to all jute serialization calls; I've seen it
when a client is trying to send a setWatches packet during reconnection,
because the client has enough watches set to exceed the 1MB limit.

:\

cc

On Sun, Sep 13, 2015 at 2:24 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:

> Why all client has to set this limit. If on zookeeper side we have
> configured this then it should use that. No?
>
> My read of the code shows that the clients also use jute.maxBuffer.
> Hopefully, one of the ZK committers can comment on this. But, jute
> (see BinaryInputArchive.java) is used on both the client and the server.
>
> I don't see any methods in CuratorrFramework which takes this variable.
>
> Per the ZK docs:
>
>
> jute.maxbuffer:
>
> (Java system property:* jute.maxbuffer*)
>
> This option can only be set as a Java system property. There is no
> zookeeper prefix on it.
>
>
>
>
> On September 13, 2015 at 4:22:46 PM, Check Peck (comptechgeeky@gmail.com)
> wrote:
>
> Why all client has to set this limit. If on zookeeper side we have
> configured this then it should use that. No?
>
> Also how can we set this parameter through Curator? I don't see any
> methods in CuratorrFramework which takes this variable.
>
> On Sun, Sep 13, 2015 at 2:17 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> Yes, all clients. Curator just wraps ZooKeeper so all the same things
>> apply.
>>
>> -Jordan
>>
>>
>>
>> On September 13, 2015 at 4:14:17 PM, Check Peck (comptechgeeky@gmail.com)
>> wrote:
>>
>> I mean this code for getting children not earlier one, that is for data.
>>
>> List<String> children = client.getChildren().forPath(path);
>>
>> On Sun, Sep 13, 2015 at 2:10 PM, Check Peck <comptechgeeky@gmail.com>
>> wrote:
>>
>>> Ok understood that part and I was correct on the 500 children nodes
>>> znode name example right?
>>>
>>> If I can increase the jute.maxBuffer property, I do need to restart the
>>> whole cluster. Correct? Is there any problem if I increase this?
>>>
>>> On Sun, Sep 13, 2015 at 2:08 PM, Jordan Zimmerman <
>>> jordan@jordanzimmerman.com> wrote:
>>>
>>>> No, the 1MB limit will hit you there as well. If you try to get your
>>>> data (via getData() call) it will fail. Note, you can increase the value
>>>> via the jute.maxBuffer property.
>>>>
>>>> -Jordan
>>>>
>>>> On September 13, 2015 at 4:06:48 PM, Check Peck (
>>>> comptechgeeky@gmail.com) wrote:
>>>>
>>>> What about data in those znode? They can be more than 1MB in each znode
>>>> correct?
>>>>
>>>>
>>>
>>
>


-- 
Christopher COORD Chen <cchen@twitter.com> @secretasianman

Mime
View raw message