hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-3638) Cache the iFile index files in memory to reduce seeks during map output serving
Date Fri, 19 Sep 2008 08:12:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632575#action_12632575
] 

chris.douglas edited comment on HADOOP-3638 at 9/19/08 1:11 AM:
----------------------------------------------------------------

This looks good. The only change would be to error handling in the acquisition:
{code}

try { 
  tmp = IndexRecord.readIndexFile(indexFileName, conf);
} catch (Throwable e) { 
  tmp = new IndexRecord[0];
  cache.remove(mapId);
} finally { 
  synchronized (newInd) { 
    newInd.indexRecordArray = tmp;
    newInd.notifyAll();
  } 
} 
{code}

So errors in loading the table don't cause other threads blocked on them to wait indefinitely.
They'll error out with ArrayIndexOutOfBoundsException, but they'll exit.

[ edit - exception name ]

      was (Author: chris.douglas):
    This looks good. The only change would be to error handling in the acquisition:
{code}

try { 
  tmp = IndexRecord.readIndexFile(indexFileName, conf);
} catch (Throwable e) { 
  tmp = new IndexRecord[0];
  cache.remove(mapId);
} finally { 
  synchronized (newInd) { 
    newInd.indexRecordArray = tmp;
    newInd.notifyAll();
  } 
} 
{code}

So errors in loading the table don't cause other threads blocked on them to wait indefinitely.
They'll error out with IndexArrayOutOfBoundsException, but they'll exit.
  
> Cache the iFile index files in memory to reduce seeks during map output serving
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-3638
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3638
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Devaraj Das
>            Assignee: Jothi Padmanabhan
>             Fix For: 0.19.0
>
>         Attachments: hadoop-3638-v1.patch, hadoop-3638-v2.patch, hadoop-3638-v3.patch,
hadoop-3638-v4.patch, hadoop-3638-v5.patch, hadoop-3638-v6.patch, hadoop-3638-v7.patch
>
>
> The iFile index files can be cached in memory to reduce seeks during map output serving.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message