chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] Created: (CHUKWA-441) Real time hadoop activity monitoring capabilities via Client Trace Logs
Date Wed, 30 Dec 2009 00:02:29 GMT
Real time hadoop activity monitoring capabilities via Client Trace Logs
-----------------------------------------------------------------------

                 Key: CHUKWA-441
                 URL: https://issues.apache.org/jira/browse/CHUKWA-441
             Project: Hadoop Chukwa
          Issue Type: New Feature
          Components: data collection, User Interface
         Environment: Redhat EL 5.1, Java 6
            Reporter: Eric Yang
            Assignee: Eric Yang
             Fix For: 0.4.0


Many moons ago, in one of the HUG talk for Chukwa, one user ask if it is possible to create
a splitter interface for Chukwa Collector for monitoring data at near real time.  The answer
was possible, but there was no code to show this capabilities.  Over the Xmas holidays, I
decided to take on this task and write something simple to show how this is done.

The idea is to reuse existing Chukwa Collector with PipelineStageWriter coupled with SocketTeeWriter
for data stream split.  By adding a SocketDataLoader inside HICC to act as data stream subscriber
and extract client trace information to render on the browser.  Hence, the additional pipeline
looks like this:

Adaptor -> Agent -> Collector -> SocketTeeWriter <- Socket Data Loader <- HICC


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message