zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rakesh Radhakrishnan <rakeshr.apa...@gmail.com>
Subject Re: Asynchronous API's and monotonicity
Date Sat, 07 Jun 2014 05:08:40 GMT
Hi Mudit,

Thanks Flavio for showing BookKeeper. Yeah this is another option you can
explore in your free time:)


I'd like to introduce it briefly to you, please have a look if you are
interested.

Documentation available at
http://zookeeper.apache.org/bookkeeper/docs/r4.2.2/

Basic terminologies:-

 - servers called as"bookies",
 - log are "ledgers",
 - and each unit of a log (aka record) is a "ledger entry"

Bookie server uses filesystem to store the ledgers and their entries(Since
it uses filesystem, message size won't be a constraint I guess). Also, it
uses ZK for storing the ledger metadata information.

Basic operations:-
1) Open a bookkeeper client.
2) Create a ledger -
Here it will internally generate id for this ledger and it will be unique.
Like ZooKeeper sequential znodes it internally maintains sequence to
generate the ids.
Upon creating a ledger, a BookKeeper client writes metadata about the
ledger to ZK.
3) Write to the ledger - User can add entries(user data) to the ledger. BK
guarantees single writer.
4) After write close the ledger.

Assume user has created four ledgers, now ledger id looks like L00001,
L00002, L00003, L00004. When user tries to create a new ledger, then id
will be incremented L00005.

Ledger metadata in ZK:
/ledgers/L00001
/ledgers/L00002
/ledgers/L00003
/ledgers/L00004
/ledgers/L00005

Now using this sequential ledger znodes present in ZK, one can write the
logic of distributed queue.

Any queries feel free to ping us happy to help you:)
You can reach us bookkeeper-dev@zookeeper.apache.org or user mailing id.

Regards,
Rakesh


On Sat, Jun 7, 2014 at 2:32 AM, Flavio Junqueira <
fpjunqueira@yahoo.com.invalid> wrote:

> You may want to have a look at BookKeeper.
>
> -Flavio
>
> On 05 Jun 2014, at 16:32, Mudit Verma <mudit.f2004912@gmail.com> wrote:
>
> > Hi Zookeeper Users,
> >
> > Lately, I have been working on a research project where I want to use
> zookeeper as a distributed logging service.
> >
> > I want to build a queue on top of zookeeper (also provided in recipes).
> >
> > What for:
> > Intention is to insert some operations performed by different clients in
> a distributed queue, and process them lazily at some later point of time.
>  And I want some ordering between these operations.
> >
> > Setup:
> > 5 physical  zookeeper servers
> >
> > The problem is:
> > In my current setup, I am observing a latency of about 13 ms per enqueue
> operation (using synchronous create APIs with sequential flags). I want to
> significantly reduce this time. The other way could be to use asynchronous
> zookeeper calls  but I am not sure what can be the side effects. Would it
> still be monotonous when used with SEQUENTIAL flag?
> >
> > For example, a  client X created a SEQUENTIAL node Z1 at time t1 using
> async create, same client created another SEQUENTIAL node Z2 at time t2
> where t2 > t1. Would the monotonic number associated with Z1 be lesser than
> that of Z2?
> >
> > Your help is much appreciated.
> >
> > Thanks
> > Mudit
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message