chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-102) Provide streaming adapter
Date Tue, 04 Aug 2009 19:02:14 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739094#action_12739094
] 

Jerome Boulon commented on CHUKWA-102:
--------------------------------------

I'm currently working on a streaming solution for Chukwa but open to share the work ;-)
Just let me know what you have in mind.

My current plan is to use Thrift for 2 principal reasons.
1- A lot of people are forced to use scribe since there's no other option and for a lot of
them, this will be the only part of their system in C. Using Thrift will provide an easy migration
path for them and make Chukwa even more attractive.
2- Thrift is working relatively well and that will save us a lot of time/energy to get a stable
version

So I'm planning on adding:
- a new Log4Appender
- a new Thrift server that can run in standalone mode or embedded inside the current collector
- Thrift will be used only for the transport, and current writer will still be responsible
for writing the data to Seq. File.
- A new Writer, that could be used to write data cross network to an another Thrift server/collector



> Provide streaming adapter
> -------------------------
>
>                 Key: CHUKWA-102
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-102
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: input tools
>            Reporter: Mac Yang
>
> Enable applications to send data directly to the agent without writing to disk first.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message