kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jibran Saithi <jib...@qubitdigital.com>
Subject Re: S3 Archiving for Kafka topics (with Zookeeper resume)
Date Thu, 31 Jan 2013 17:36:53 GMT
I should probably call it something else to avoid confusion, it started as
a fork but since became a re-write (the linked one was using simpleconsumer
and we needed some Zookeeper goodness).



On 31 January 2013 17:04, Jay Kreps <jay.kreps@gmail.com> wrote:

> This is super cool. Are we linking the right repo on the ecosystem page?
>
> https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem
>
> -Jay
>
>
> On Thu, Jan 31, 2013 at 1:29 AM, Jibran Saithi <jibran@qubitdigital.com
> >wrote:
>
> > Hey,
> >
> > I know this has come up a few times, so thought I'd share a bit of code
> > we've been using to archive topics to S3.
> >
> > Particularly unimaginatively named, but is available here:
> > https://github.com/jibs/kafka-s3-consumer
> >
> > We needed something with Zookeeper support for storing the offsets, but
> > didn't come across anything so I quickly put this together. For the
> moment
> > I've removed graphite stats reporting because it has a few internal
> > dependencies, but plan to sort that out soon.
> >
> > Hope this helps,
> > Jibran
> >
>



-- 
Jibran Saithi
Product Engineer


QuBit Digital Ltd
www.qubitdigital.com
*
*
Office: +44 (0)203 411 9130

QuBit Digital Ltd
20 Broadwick Street
London
W1F 8HT



This email may be confidential or privileged. If you received this
communication by mistake, please don't forward it to anyone else, please
erase all copies and attachments, and please let me know that it has gone to
the wrong person. Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message