kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From San <sahmed1...@gmail.com>
Subject Re: S3 Archiving for Kafka topics (with Zookeeper resume)
Date Thu, 31 Jan 2013 17:54:35 GMT
I see the main method, once you run this will it stop execution or it will keep waiting and
processing is what I mean?

Supervisord from what I understand is used to ensure the process stats up after reboots and
restarts if it crashes etc

Sent from my iPhone

On 2013-01-31, at 12:38 PM, Jibran Saithi <jibran@qubitdigital.com> wrote:

> Yeah, we use supervisor which seems to work fine for us.
> 
> On 31 January 2013 14:57, S Ahmed <sahmed1020@gmail.com> wrote:
> 
>> Great thanks.
>> 
>> BTW, that's not a daemon server is it?  Is it something you have to wrap to
>> run as a service/daemon?
>> 
>> 
>> On Thu, Jan 31, 2013 at 4:29 AM, Jibran Saithi <jibran@qubitdigital.com
>>> wrote:
>> 
>>> Hey,
>>> 
>>> I know this has come up a few times, so thought I'd share a bit of code
>>> we've been using to archive topics to S3.
>>> 
>>> Particularly unimaginatively named, but is available here:
>>> https://github.com/jibs/kafka-s3-consumer
>>> 
>>> We needed something with Zookeeper support for storing the offsets, but
>>> didn't come across anything so I quickly put this together. For the
>> moment
>>> I've removed graphite stats reporting because it has a few internal
>>> dependencies, but plan to sort that out soon.
>>> 
>>> Hope this helps,
>>> Jibran
> 
> 
> 
> -- 
> Jibran Saithi
> Product Engineer
> 
> 
> QuBit Digital Ltd
> www.qubitdigital.com
> *
> *
> Office: +44 (0)203 411 9130
> 
> QuBit Digital Ltd
> 20 Broadwick Street
> London
> W1F 8HT
> 
> 
> 
> This email may be confidential or privileged. If you received this
> communication by mistake, please don't forward it to anyone else, please
> erase all copies and attachments, and please let me know that it has gone to
> the wrong person. Thanks

Mime
View raw message