incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (HAMA-387) Add task ID and superstep count informations to lock file
Date Tue, 24 May 2011 10:18:47 GMT

    [ https://issues.apache.org/jira/browse/HAMA-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038473#comment-13038473
] 

Thomas Jungblut edited comment on HAMA-387 at 5/24/11 10:16 AM:
----------------------------------------------------------------

Won't work. 

{noformat}
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
= NoNode for /bsp/1_thomas-desktop:56492
{noformat}

ZooKeeper sucks?:D

EDIT:
We actually have to set the superstep count into the byte value of this lock. Then we have
to get the object and deserialize it then to check in which superstep the node is...

{noformat}

private int countGroomsInSuperStep(List<String> list, long superStep) throws KeeperException,
InterruptedException{
    int count = 0;
    for(String groom : list){
      byte[] data = zk.getData(bspRoot + "/" + groom, null, null);
      if(Bytes.toLong(data) == superStep)
        count++;
    }
    return count;
  }

{noformat}

And the loop is then going like:
{noformat}
 while (true) {
      synchronized (mutex) {
        List<String> list = zk.getChildren(bspRoot, true);
        if (countGroomsInSuperStep(list,this.getSuperstepCount()) > 0) {
          mutex.wait();
        } else {
          LOG.debug("[" + getPeerName() + "] leave from the leaveBarrier");
          return true;
        }
      }
    }
{noformat}

      was (Author: thomas.jungblut):
    Won't work. 

{noformat}
java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode
= NoNode for /bsp/1_thomas-desktop:56492
{noformat}

ZooKeeper sucks?:D

EDIT:
We actually have to set the superstep count into the byte value of this lock. Then we have
to get the object and deserialize it then to check in which superstep the node is...

This is really crappy, we should open a feature ticket on the ZK project.
  
> Add task ID and superstep count informations to lock file
> ---------------------------------------------------------
>
>                 Key: HAMA-387
>                 URL: https://issues.apache.org/jira/browse/HAMA-387
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp
>    Affects Versions: 0.2.0
>            Reporter: Edward J. Yoon
>             Fix For: 0.3.0
>
>         Attachments: sleepless.patch
>
>
> I think, the lock file must include:
>  * the job ID
>  * the task ID of the lock file owner
>  * the current superstep count
> to check ownership and validation.
> Currently they are named by hostname, but multi-tasks can be run per one groomserver
in the future. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message