giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-912) support succinct representation of messages in messagestores
Date Thu, 10 Jul 2014 03:15:05 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057079#comment-14057079
] 

Hudson commented on GIRAPH-912:
-------------------------------

SUCCESS: Integrated in Giraph-trunk-Commit #1462 (See [https://builds.apache.org/job/Giraph-trunk-Commit/1462/])
GIRAPH-912: Support succinct representation of messages in messagestores (pavanka) (pavanka:
http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=f31e9a328d3b4f10906a10a8e69d2ae515e3aba0)
* giraph-core/src/main/java/org/apache/giraph/utils/VertexIdMessageIterator.java
* giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerOneMessageToManyRequest.java
* giraph-core/src/main/java/org/apache/giraph/comm/SendMessageToAllCache.java
* giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayOneToAllMessages.java
* giraph-core/src/main/java/org/apache/giraph/comm/requests/RequestType.java
* giraph-core/src/main/java/org/apache/giraph/utils/UnsafeReusableByteArrayInput.java
* giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/AbstractListPerVertexStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/long_id/LongPointerListMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/utils/UnsafeReads.java
* giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
* giraph-core/src/main/java/org/apache/giraph/utils/OneMessageToManyIdsIterator.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/MessageStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/MessageEncodeAndStoreType.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/IntByteArrayMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/InMemoryMessageStoreFactory.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/OneMessagePerVertexStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/long_id/LongAbstractMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/long_id/LongByteArrayMessageStore.java
* giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java
* giraph-core/src/main/java/org/apache/giraph/utils/VertexIdDataIterator.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/PointerListPerVertexStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/long_id/package-info.java
* giraph-core/src/main/java/org/apache/giraph/utils/ExtendedByteArrayOutputBuffer.java
* giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdDataIterator.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/out_of_core/DiskBackedMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/utils/ByteStructVertexIdMessageIterator.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/IntFloatMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
* giraph-core/src/main/java/org/apache/giraph/utils/ExtendedDataOutput.java
* giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerOneToAllMessagesRequest.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/ByteArrayMessagesPerVertexStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/long_id/LongAbstractListMessageStore.java
* giraph-core/src/test/java/org/apache/giraph/comm/messages/TestLongDoublePrimitiveMessageStores.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/LongDoubleMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/PointerListMessagesIterable.java
* giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/LongByteArrayMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/utils/ByteArrayOneMessageToManyIds.java
* giraph-core/src/main/java/org/apache/giraph/utils/UnsafeArrayReads.java
* giraph-core/src/main/java/org/apache/giraph/comm/SendOneMessageToManyCache.java
* CHANGELOG


> support succinct representation of messages in messagestores
> ------------------------------------------------------------
>
>                 Key: GIRAPH-912
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-912
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Pavan Kumar
>            Assignee: Pavan Kumar
>         Attachments: GIRAPH-912.patch
>
>
> currently one-to-all messages is useful only for reducing network transfer, but at the
server side, before processingRequest, the messages are broken down into bytearrayvertexidmessages,
thus, each message is still stored as many times as the number of vertices it is sent to.

> in this task we implements two message-stores that can represent messages in a very succinct
way
> especially useful when messages are of significant size & fanout is low [note that
this is - useful only when no combiners are used]
> this depends on GIRAPH-907 & GIRAPH-908



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message