hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ananda Verma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7821) Hadoop event notification system
Date Sat, 23 Mar 2013 03:17:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13611538#comment-13611538

Ananda Verma commented on HADOOP-7821:

We are building a "Distributed Hadoop Event Notification System" using ActiveMQ pubsub model.
We are working towards building a event notification based job scheduler which trigger job
based on certain event in HDFS e.g. data availability. Also multiple map-reduce jobs can be
chained in this manner. Following other issue can be solved:

1. NameNode polling by different hadoop clients can be avoided.
2. Jobs can be controlled by various events e.g. execute the next job only if cluster overall
load is below certain level.
3. Provide a 'customizable' and 'configurable' interface such that user-defined parameters
can also be 'subscribed' by the user
4. Provide timely state/status/statistics updates to subscribed users.

> Hadoop event notification system
> --------------------------------
>                 Key: HADOOP-7821
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7821
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 0.24.0
>            Reporter: John George
> It will be good if Hadoop supports some sort of a messaging service that allows users
to subscribe.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message