hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4342) Distributed Cache gives inconsistent result if cache files get deleted from task tracker
Date Mon, 30 Jul 2012 20:49:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425225#comment-13425225
] 

Alejandro Abdelnur commented on MAPREDUCE-4342:
-----------------------------------------------

Mayank, I don't see my previous second comment being addressed. I was meaning the following.

Instead doing:

{code}
  public void handle(ResourceEvent event) {
    LocalResourceRequest req = event.getLocalResourceRequest();
    LocalizedResource rsrc = localrsrc.get(req);
    if (rsrc != null
        && (event.getType() == ResourceEventType.LOCALIZED || event.getType() == ResourceEventType.REQUEST)
        && (!isResourcePresent(rsrc))) {
      LOG.info("Resource " + rsrc.getLocalPath()
          + " is missing, localizing it again");
      localrsrc.remove(req);
      rsrc = null;
    }
    switch (event.getType()) {
    case REQUEST:
    case LOCALIZED:
      if (null == rsrc) {
        rsrc = new LocalizedResource(req, dispatcher);
        localrsrc.put(req, rsrc);
      }
      break;
    ....
{code}

Do:

{code}
  public void handle(ResourceEvent event) {
    LocalResourceRequest req = event.getLocalResourceRequest();
    LocalizedResource rsrc = localrsrc.get(req);
    switch (event.getType()) {
    case REQUEST:
    case LOCALIZED:
      if (rsrc != null && !isResourcePresent(rsrc)) {
        LOG.info("Resource " + rsrc.getLocalPath()
            + " is missing, localizing it again");
        localrsrc.remove(req);
        rsrc = null;
      }
      if (null == rsrc) {
        rsrc = new LocalizedResource(req, dispatcher);
        localrsrc.put(req, rsrc);
      }
      break;
    ....
{code}


                
> Distributed Cache gives inconsistent result if cache files get deleted from task tracker

> -----------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4342
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4342
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.22.0, 1.0.3, trunk
>            Reporter: Mayank Bansal
>            Assignee: Mayank Bansal
>         Attachments: MAPREDUCE-4342-22-1.patch, MAPREDUCE-4342-22-2.patch, MAPREDUCE-4342-22-3.patch,
MAPREDUCE-4342-22.patch, MAPREDUCE-4342-trunk-v2.patch, MAPREDUCE-4342-trunk.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message