incubator-giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-185) Improve concurrency of putMsg / putMsgList
Date Tue, 24 Apr 2012 21:34:07 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261046#comment-13261046
] 

jiraposter@reviews.apache.org commented on GIRAPH-185:
------------------------------------------------------



bq.  On 2012-04-24 20:53:33, Avery Ching wrote:
bq.  > http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java,
lines 776-777
bq.  > <https://reviews.apache.org/r/4852/diff/1/?file=104059#file104059line776>
bq.  >
bq.  >     Bo, I'm a little leery about converting the List and ArrayList to LinkedList
and ConcurrentLinkedList.  I believe that linked list's will use more memory than the array
list due to the double links (forward and backward).  Also, is ConcurrentLinkedList supposted
to outperform a synchronized ArrayList?  I haven't seen much on that.
bq.  >     
bq.  >     The concurrenthashmap changes look good.
bq.  
bq.  Bo Wang wrote:
bq.      Avery, thanks for the comments. I just measured the sizes of these classes and below
are an estimation. 
bq.      
bq.      java.util.ArrayList: 149 bytes
bq.      java.util.LinkedList: 101 bytes
bq.      java.util.concurrent.ConcurrentLinkedQueue: 118 bytes
bq.      
bq.      The tool I was using is a program from the link below.
bq.      http://www.javapractices.com/topic/TopicAction.do?Id=83
bq.      
bq.      In terms of performance, here is a benchmark.
bq.      http://www.javacodegeeks.com/2010/09/java-best-practices-queue-battle-and.html
bq.      
bq.      In its test #1 (adding element), ConcurrentLinkedQueue performed slightly better
than LinkedList. In test #3 (iterator), LinkedList outperformed ConcurrentLinkedQueue. I think
the most time consuming part is add, while iteration is also heavily used but no concurrent
accesses. 
bq.      
bq.

Thanks for the response Bo.

Those numbers are for the empty data structures I'm assuming.  I was referring to the incremental
cost of adding elements (messages) to the data structures.  The performance isn't a a concern
to me (unless we call size() somewhere).


- Avery


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4852/#review7185
-----------------------------------------------------------


On 2012-04-24 06:11:38, Bo Wang wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4852/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-04-24 06:11:38)
bq.  
bq.  
bq.  Review request for giraph.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Use ConcurrentHashMap and ConcurrentLinkedQueue to allow concurrent assess to message
map. The concurrencyLevel of ConcurrentHashMap uses the default value. There may be some performance
gain by tuning this value.
bq.  
bq.  
bq.  This addresses bug GIRAPH-185.
bq.      https://issues.apache.org/jira/browse/GIRAPH-185
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
1328747 
bq.  
bq.  Diff: https://reviews.apache.org/r/4852/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Bo
bq.  
bq.


                
> Improve concurrency of putMsg / putMsgList
> ------------------------------------------
>
>                 Key: GIRAPH-185
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-185
>             Project: Giraph
>          Issue Type: Improvement
>          Components: graph
>    Affects Versions: 0.2.0
>            Reporter: Bo Wang
>            Assignee: Bo Wang
>             Fix For: 0.2.0
>
>         Attachments: GIRAPH-185.patch, GIRAPH-185.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> Currently in putMsg / putMsgList, a synchronized closure is used to protect the whole
transientInMessages when adding the new message. This lock prevents other concurrent calls
to putMsg/putMsgList and increases the response time. We should use fine-grain locks to allow
high concurrency in message communication.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message