hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1491) Use HAR filesystem to merge parity files
Date Tue, 16 Feb 2010 23:39:27 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

dhruba borthakur updated MAPREDUCE-1491:
----------------------------------------

       Resolution: Fixed
    Fix Version/s: 0.22.0
     Hadoop Flags: [Reviewed]
           Status: Resolved  (was: Patch Available)

Thanks for the explanation Rodrigo. I was successfully able to run the unit test on a fresh
checkout of my tree. I have committed this. 

> Use HAR filesystem to merge parity files 
> -----------------------------------------
>
>                 Key: MAPREDUCE-1491
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1491
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/raid
>            Reporter: Rodrigo Schmidt
>            Assignee: Rodrigo Schmidt
>             Fix For: 0.22.0
>
>         Attachments: MAPREDUCE-1491.0.patch
>
>
> The HDFS raid implementation (HDFS-503) creates a parity file for every file that is
RAIDed. This puts additional burden on the memory requirements of the namenode. It will be
 nice if the parity files are combined together using the HadoopArchive (har) format.
> This was (HDFS-684) before, but raid migrated to MAPREDUCE.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message