zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian Fang <jian.fang.subscr...@gmail.com>
Subject Questions about Book Keeper
Date Wed, 11 Jan 2012 20:02:09 GMT

I am pretty new to Zoo keeper and book keeper.

Right now, I am implementing a distributed data processing pipeline, which
includes multiple stages. The input is a stream of product entries. I like
to use Book keeper to do write ahead logging for the input to the pipeline
so that if some process dies, the pipeline can recover from the server
failure simply by reading the unprocessed log. What I want is to first
write each input product info into the log using book keeper and mark or
delete the product info once the pipeline finishes processing the product.
I am not sure how to apply book keeper for this scenario, more
specifically,  how to mark or delete the log after the product is out of
the pipeline.

Also, I wonder if I can embed the book keeper server inside my application
to reduce the operational overhead.

Your help is highly appreciate.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message