hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milind Parikh <milindpar...@gmail.com>
Subject Re: is zookeeper suitable for my application?
Date Fri, 08 Oct 2010 15:18:20 GMT
There are existing recipes that you will be able to modify for your needs.
Ymmv but look at the double barrier recipe. The key thing to design around
in high. Scalability based solution is the herding effect.
Perhaps look @ couchdb to store your files for znode?
Regards
Milind

On Oct 8, 2010 3:31 AM, "Li Li" <fancyerii@gmail.com> wrote:

hi all,
   I am a newbie of zookeeper and want to know whether zookeeper is
suitable for my application. If it does, then I will try to learn more
about it.
   My problem can be described as a producer-consumer problem.
   There is a thread that continue producing data files untill the
queue is full and then waiting consumer. Consumers can be divided into
2 categories -- Master and Slave. The data can not be removed from
queue unless it's consumed by master. The data will be removed
immediately if it consumed by all the consumers. But if the queue is
full, it will remove data that are consumed by Master. There will be
only one Master and many Slaves. The count of Slaves will change(it
seems zookeeper provide Ephemeral ZNode).
   it seems that znode's size should less than 1 mb. but I will need
save large data file in a znode. Is there any solution for this?
   thank you.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message