hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MaoYuan Xian (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HAMA-756) Timing issue and file merging algorithm in PartitioningRunner make job fail
Date Fri, 10 May 2013 03:49:15 GMT
MaoYuan Xian created HAMA-756:
---------------------------------

             Summary: Timing issue and file merging algorithm in PartitioningRunner make job
fail
                 Key: HAMA-756
                 URL: https://issues.apache.org/jira/browse/HAMA-756
             Project: Hama
          Issue Type: Bug
            Reporter: MaoYuan Xian
            Assignee: MaoYuan Xian


There are two major problems in bsp methor of PartitioningRunner may make the partitioning
fail:
1. The call to peer.getNumPeers() may trigger the timing issue. In the special situation when
some tasks complete the bsp call but some others just enter the "for (FileStatus statu : status)"
loop, these remaining task calling to peer.getNumPeers() will trigger the problem.
2. The algorithm of merging the sequence files has the problem: e.g. when desiredNum is 8
and partitioning task number (peer.getNumPeers()) is 6, the part-7 directory can not find
the handler to merging it as a file.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message