hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharad Agarwal (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-814) Move completed Job history files to HDFS
Date Fri, 31 Jul 2009 07:44:14 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sharad Agarwal updated MAPREDUCE-814:
-------------------------------------

    Attachment: 814_v1.patch

An early patch:
-Moves the history files to HDFS in a separate thread. I have used a thread pool so that the
movement of files are not stalled for long time in case of datanodes being slow.
-Users can provide the location of the done folder using "mapred.job.tracker.history.completed.location"
-Testing in progress. Test case not added yet.


> Move completed Job history files to HDFS
> ----------------------------------------
>
>                 Key: MAPREDUCE-814
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-814
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: jobtracker
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>         Attachments: 814_v1.patch
>
>
> Currently completed job history files remain on the jobtracker node. Having the files
available on HDFS will enable clients to access these files more easily.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message