hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharad Agarwal (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-814) Move completed Job history files to HDFS
Date Fri, 31 Jul 2009 07:44:14 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sharad Agarwal updated MAPREDUCE-814:

    Attachment: 814_v1.patch

An early patch:
-Moves the history files to HDFS in a separate thread. I have used a thread pool so that the
movement of files are not stalled for long time in case of datanodes being slow.
-Users can provide the location of the done folder using "mapred.job.tracker.history.completed.location"
-Testing in progress. Test case not added yet.

> Move completed Job history files to HDFS
> ----------------------------------------
>                 Key: MAPREDUCE-814
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-814
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: jobtracker
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>         Attachments: 814_v1.patch
> Currently completed job history files remain on the jobtracker node. Having the files
available on HDFS will enable clients to access these files more easily.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message