giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Avery Ching (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-389) Multithreading should intelligently allocate the thread pools
Date Sat, 27 Oct 2012 15:17:11 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485451#comment-13485451
] 

Avery Ching commented on GIRAPH-389:
------------------------------------

https://reviews.apache.org/r/7754/

 Description:

Do not create more input split loading threads than input splits to allow workers to equally
load up input splits
Do not create more compute threads than partitions on the worker
Removed duplicate reserveInputSplit method in BspServiceWorker
Minor optimization to only get the input split zknodes once and pass to all threads.

Testing Done:

passed unittests
ran pagerankbenchmark on a real cluster and observed that the limiting works


                
> Multithreading should intelligently allocate the thread pools
> -------------------------------------------------------------
>
>                 Key: GIRAPH-389
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-389
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Avery Ching
>
> Even if the user suggests a very high number of threads, the input split threads should
not exceed the number splits divided by the number of  workers.
> The number of compute threads should not be greater than the number of partitions on
that worker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message