hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "paul sutter (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-51) per-file replication counts
Date Fri, 07 Apr 2006 01:47:32 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-51?page=comments#action_12373565 ] 

paul sutter commented on HADOOP-51:

+1 that

it might be easier to use on a per-directory basis, examples:

- /tmp directory, replication count 2 (or 1!), a good place for the output of intermediate
reduce steps
- /cached directory, infinite replicaton count, a good place for lookup files used in mappers
or reducers

> per-file replication counts
> ---------------------------
>          Key: HADOOP-51
>          URL: http://issues.apache.org/jira/browse/HADOOP-51
>      Project: Hadoop
>         Type: New Feature

>   Components: dfs
>     Versions: 0.2
>     Reporter: Doug Cutting
>     Assignee: Konstantin Shvachko
>      Fix For: 0.2

> It should be possible to specify different replication counts for different files.  Perhaps
an option when creating a new file should be the desired replication count.  MapReduce should
take advantage of this feature so that job.xml and job.jar files, which are frequently accessed
by lots of machines, are more highly replicated than large data files.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message