hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-842) Add persistent queue option to JobConf
Date Tue, 14 Jan 2014 00:07:57 GMT

    [ https://issues.apache.org/jira/browse/HAMA-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13870156#comment-13870156
] 

Hudson commented on HAMA-842:
-----------------------------

SUCCESS: Integrated in Hama-Nightly-for-Hadoop-1.x #1150 (See [https://builds.apache.org/job/Hama-Nightly-for-Hadoop-1.x/1150/])
HAMA-842: Add persistent queue option to JobConf (edwardyoon) (edwardyoon: rev 1557644)
* /hama/trunk/CHANGES.txt
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJob.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/AbstractMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/MessageTransferQueueFactory.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/DiskQueue.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MemoryQueue.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/MessageQueue.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SingleLockQueue.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SortedMessageQueue.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/queue/SpillingQueue.java
* /hama/trunk/core/src/test/java/org/apache/hama/bsp/TestPersistQueue.java
* /hama/trunk/examples/src/test/java/org/apache/hama/examples/BipartiteMatchingTest.java


> Add persistent queue option to JobConf
> --------------------------------------
>
>                 Key: HAMA-842
>                 URL: https://issues.apache.org/jira/browse/HAMA-842
>             Project: Hama
>          Issue Type: New Feature
>    Affects Versions: 0.6.3
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 0.7.0
>
>         Attachments: HAMA-842_v01.patch, HAMA-842_v02.patch
>
>
> By default, incoming (local) queue will be reset in every superstep. It means that unconsumed
messages will be lost. If this is persistent queue, unconsumed messages will be persisted
in incoming queue.
> {code}
>   public final void clearOutgoingQueues() {
>     if (localQueue != null) {
>       localQueue.close();
>     }
>     localQueue = localQueueForNextIteration.getMessageQueue();
>     localQueue.prepareRead();
>     localQueueForNextIteration = getSynchronizedReceiverQueue();
>     notifyInit();
>   }
> {code}
> Simply we can add unconsumed messages to localQueueForNextIteration before switching
local queue.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message