chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Graham (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CHUKWA-477) Support post-demux triggers
Date Tue, 20 Apr 2010 09:39:10 GMT

     [ https://issues.apache.org/jira/browse/CHUKWA-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bill Graham updated CHUKWA-477:
-------------------------------

    Attachment: CHUKWA-477.1.patch

Here's a first pass at this functionality (CHUKWA-477.1.patch) for review. Please review and
provide feedback re design and naming. A few things to note:

- I've added the following constant and interface/package:
{noformat}
package org.apache.hadoop.chukwa.datatrigger;

public interface TriggerAction {

  public void load(ChukwaConfiguration conf, FileSystem fs, FileStatus[] src) throws IOException;
}

public interface CHUKWA_CONSTANT {
  public static final String POST_DEMUX_SUCCESS_ACTION = "chukwa.post.demux.success.action";
 
}
{noformat}

- The idea is that you can implement specific TriggerActions for to occur in different scenarios.
They know how to react to some action, which is relayed via the FileStatus[] objects. The
method signature of TriggerAction is the same as that of DataLoaderFactory, so the latter
could implement the former if we want to standardize and go that route. Or we could create
a new signature for TriggerAction that contains more context re the trigger itself.

- PostProcessorMananger does not know the names of the files that were added to repos/, since
those details are never returned by MoveToRepository. I'd like to change MoveToRepository
to return a collection of Paths to the action can see what's been added. This could be done
in a separate JIRA if that makes more sense.

> Support post-demux triggers
> ---------------------------
>
>                 Key: CHUKWA-477
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-477
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: Data Processors
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-477.1.patch
>
>
> Add the ability to trigger an action upon successful completion of a Demux process. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message