chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <>
Subject [jira] Commented: (CHUKWA-102) Provide streaming adapter
Date Tue, 04 Aug 2009 19:02:14 GMT


Jerome Boulon commented on CHUKWA-102:

I'm currently working on a streaming solution for Chukwa but open to share the work ;-)
Just let me know what you have in mind.

My current plan is to use Thrift for 2 principal reasons.
1- A lot of people are forced to use scribe since there's no other option and for a lot of
them, this will be the only part of their system in C. Using Thrift will provide an easy migration
path for them and make Chukwa even more attractive.
2- Thrift is working relatively well and that will save us a lot of time/energy to get a stable

So I'm planning on adding:
- a new Log4Appender
- a new Thrift server that can run in standalone mode or embedded inside the current collector
- Thrift will be used only for the transport, and current writer will still be responsible
for writing the data to Seq. File.
- A new Writer, that could be used to write data cross network to an another Thrift server/collector

> Provide streaming adapter
> -------------------------
>                 Key: CHUKWA-102
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: input tools
>            Reporter: Mac Yang
> Enable applications to send data directly to the agent without writing to disk first.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message