Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CANykduZorvbfgxquJvNeE+tnXCz6XJa529FZ0VYczo+4qiNWJg@mail.gmail.com>
References: 
 <CAF+A=7rfiwAP-sAGUMGRzGF15cLA4D9cQmP=Ra_XUUCFU_ceJA@mail.gmail.com>
 <CANykdua1XfTL-351ZtJZfP8Z0oO-dfec=P9CKjnz5oXF1FRYkA@mail.gmail.com>
 <CAF+A=7p1hZtfax4_Gu32r4a-Q3pmtrrSP47RKAOmU2QX_Sp2ZQ@mail.gmail.com>
 <CANykduZorvbfgxquJvNeE+tnXCz6XJa529FZ0VYczo+4qiNWJg@mail.gmail.com>
From: Check Peck <comptechgeeky@gmail.com>
Date: Sun, 13 Sep 2015 13:47:00 -0700
Message-ID: 
 <CAF+A=7pZ2FC6pOzBwN=BDHQcgBiRk2uAX1334e800ugOBn4fcQ@mail.gmail.com>
Subject: Re: zookeeper design problem
To: user <user@zookeeper.apache.org>
Content-Type: multipart/alternative; boundary=001a113475ae18b566051fa70e94

--001a113475ae18b566051fa70e94
Content-Type: text/plain; charset=UTF-8

Ok understood. As per other thread in which Jordan mention to me this:


*In terms of length, ZooKeeper has a 1MB limit per API call. So, for
example, if you call getChildren(), the entire length of all the children
can't exceed 1MB (unless you reconfigure ZK).*
In my case, what does it mean? If I have 1000 clients and each client has
100 znodes and those znodes have some data in it. Then what will fail if I
try to do something?

I am trying to get a better idea on this.

On Sat, Sep 12, 2015 at 11:22 PM, Cameron McKenzie <mckenzie.cam@gmail.com>
wrote:

> You can call the event / timestamp zNode's whatever you like. I'd just use
> a persistent sequential node for the name, and have your reaper make its
> decisions based on the creation / modification time of the event /
> timestamp zNode that you're referring to rather than the name of the zNode
> itself.
>
> On Sun, Sep 13, 2015 at 4:03 PM, Check Peck <comptechgeeky@gmail.com>
> wrote:
>
> > Thanks Cameron for suggestion. Is it ok if I have timestamp1 instead of
> > event1, timestamp2 instead of event2 and so on? Or do you think it might
> be
> > bad idea.
> >
> > This timestamp will be System.currentTimeMillis() from Java.
> >
> > On Sat, Sep 12, 2015 at 10:06 PM, Cameron McKenzie <
> mckenzie.cam@gmail.com
> > >
> > wrote:
> >
> > > Can't you just add a zNode under client_xxx for each of your 'event'
> > > triplets, and then have the reaper go and remove that whole child node
> > when
> > > the 'event' node is beyond a certain age? Something like:
> > >
> > > /root/clients/client_100/event1/metric
> > > /root/clients/client_100/event1/transaction
> > > /root/clients/client_100/event1/log
> > > /root/clients/client_100/event2/metric
> > > /root/clients/client_100/event2/transaction
> > > /root/clients/client_100/event2/log
> > > /root/clients/client_101/event1/metric
> > > /root/clients/client_101/event1/transaction
> > > /root/clients/client_101/event1/log
> > >
> > >
> > > On Sun, Sep 13, 2015 at 7:59 AM, Check Peck <comptechgeeky@gmail.com>
> > > wrote:
> > >
> > > > I am working on designing the zookeeper node hierarchy so looking for
> > > some
> > > > suggestion. I have a clients node under which I will have multiple
> > > clients
> > > > (for ex: client_100, client_101)
> > > >
> > > >     /root/clients/client_100
> > > >     /root/clients/client_101
> > > >     /root/clients/client_102
> > > >
> > > > Now inside each client, I need to store three things:
> > > >
> > > >    1. First is metric data which will be in JSON format (max 15
> > key-value
> > > >    pairs).
> > > >    2. Second is transaction data which will be plain simple string
> > (this
> > > >    data should be less than 1 MB).
> > > >    3. Third is client logs data which will also be plain simple
> string
> > > >    (this data should also be less than 1 MB).
> > > >
> > > > Also now most important thing, I can have multiple of above three
> > things
> > > > for each client. Meaning for client_100, I can have two metric data,
> > two
> > > > transaction data and two client logs. Here two is just a number it
> can
> > > be X
> > > > but I will have my reaper process running that can clean the old
> > "metric
> > > > data, transaction data and client logs" for each client after certain
> > > > period or some threshold.
> > > >
> > > > In short, for each client I have a list of above three things which I
> > > need
> > > > to show on zookeeper. What is the right design for this so that I
> don't
> > > > destroy zookeeper. I can have each znode for each of above three
> things
> > > but
> > > > I am not sure how can I bucket this since it will be multiple for
> each
> > > > clients.
> > > >
> > > > Also max number of clients I can have is 1000 that's all. But I will
> > have
> > > > another reaper process for this as well to keep deleting older
> clients
> > > > which doesn't had any activity from a long time.
> > > >
> > >
> >
>

--001a113475ae18b566051fa70e94--