hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2494) Make the distributed cache delete entires using LRU priority
Date Wed, 08 Jun 2011 17:26:07 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046094#comment-13046094
] 

Hadoop QA commented on MAPREDUCE-2494:
--------------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12481842/MAPREDUCE-2494-20.20X-V1.patch
  against trunk revision 1133226.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    -1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/365//console

This message is automatically generated.

> Make the distributed cache delete entires using LRU priority
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-2494
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2494
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: distributed-cache
>    Affects Versions: 0.21.0
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2494-20.20X-V1.patch, MAPREDUCE-2494-V1.patch, MAPREDUCE-2494-V2.patch
>
>
> Currently the distributed cache will wait until a cache directory is above a preconfigured
threshold.  At which point it will delete all entries that are not currently being used. 
It seems like we would get far fewer cache misses if we kept some of them around, even when
they are not being used.  We should add in a configurable percentage for a goal of how much
of the cache should remain clear when not in use, and select objects to delete based off of
how recently they were used, and possibly also how large they are/how difficult is it to download
them again.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message