hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bibin A Chundatt (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-6466) NNBench map task should create files in separate folder
Date Fri, 13 May 2016 16:43:13 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bibin A Chundatt resolved MAPREDUCE-6466.
-----------------------------------------
    Resolution: Duplicate

MAPREDUCE-6666 handling the same.

> NNBench map task should create files in separate folder
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-6466
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6466
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>
> In case of NNBench  with current implementation  all files created using different maps
are created in same directory.
> In this JIRA will provide a flag to create files in same folder or separate folder for
each task.
> {{dfs.namenode.fs-limits.max-directory-items}} cannot be set to more than
> 6,400,000 as i understand
> Advantage
> # For scenarios like creating fsimage with large size NNBench can be used
> # Creation of multiple files of same size using NNBench folder based.
> No need to run NNBench multiple time to create FSImage of large size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message