hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-5603) Ability to disable FileInputFormat listLocatedStatus optimization to save client memory
Date Tue, 22 Apr 2014 17:36:16 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jason Lowe updated MAPREDUCE-5603:
----------------------------------

          Resolution: Won't Fix
    Target Version/s: 2.3.0, 3.0.0, 0.23.11  (was: 3.0.0, 0.23.11, 2.3.0)
              Status: Resolved  (was: Patch Available)

Closing this as won't fix for now.  This was originally filed as a stop-gap in case there
were any situations where the client needed the extra memory and simply couldn't use a larger
heap.  However after running with the listLocatedStatus fix for quite some time we've only
seen a couple of instances where the client needed more memory to handle the situation and
it was an easy fix to simply increase the memory used by the client.

There's plenty of confs in Hadoop as it is, and I'd rather not add yet another conf if it's
not necessary.  We can reopen this if the need arises.

> Ability to disable FileInputFormat listLocatedStatus optimization to save client memory
> ---------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5603
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5603
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client, mrv2
>    Affects Versions: 0.23.10, 2.2.0
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Minor
>         Attachments: MAPREDUCE-5603.patch, MAPREDUCE-5603.patch
>
>
> It would be nice if users had the option to disable the listLocatedStatus optimization
in FileInputFormat to save client memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message