hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jothi Padmanabhan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3638) Cache the iFile index files in memory to reduce seeks during map output serving
Date Thu, 18 Sep 2008 14:20:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jothi Padmanabhan updated HADOOP-3638:
--------------------------------------

    Status: Patch Available  (was: Open)

Attaching a new patch based on offline comments from Chris.
We now have a read-write lock to prevent updates to timestamps when the LRU algorithm is in
effect.
Analysis of a random task tracker log towards the end of a Gridmix run indicated that the
Index-cache-memory-limit was never reached and the LRU algorithm was never invoked at all.
The log also showed that the number of IndexCache hits was about 50K and the misses were less
then 300.

The following is the result of testpatch on my local machine. Testing was done by verifying
that applications like sort and Gridmix ran successfully. Also, testing was one by varying
different parameters (for the memory limits) and verifying that all code paths were visited.

     [exec] 
     [exec] -1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     -1 tests included.  The patch doesn't appear to include any new or modified
tests.
     [exec]                         Please justify why no tests are needed for this patch.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec] 
     [exec] 


> Cache the iFile index files in memory to reduce seeks during map output serving
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3638
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3638
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.17.0
>            Reporter: Devaraj Das
>            Assignee: Jothi Padmanabhan
>             Fix For: 0.19.0
>
>         Attachments: hadoop-3638-v1.patch, hadoop-3638-v2.patch, hadoop-3638-v3.patch,
hadoop-3638-v4.patch, hadoop-3638-v5.patch, hadoop-3638-v6.patch
>
>
> The iFile index files can be cached in memory to reduce seeks during map output serving.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message