giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alessandro Presta (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (GIRAPH-462) Multithreading breaks out-of-core graph
Date Tue, 01 Jan 2013 20:30:13 GMT

     [ https://issues.apache.org/jira/browse/GIRAPH-462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alessandro Presta updated GIRAPH-462:
-------------------------------------

    Description: 
[~cmartella] pointed out this issue: when using multithreaded computation in conjunction with
out-of-core graph, we incur in a race condition. The compute threads share the same DiskBackedPartitionStore,
whose getPartition() method is not meant to be thread-safe. When two threads request two out-of-core
partitions concurrently, they both try to load it to the same slot.
The result is that we can lose the reference to one of the two partitions (which will not
be written back to disk) and we can incur in a NullPointerException when both threads are
trying to offload the currently loaded partition to disk.

I ran this test to confirm the issue:
https://gist.github.com/4429628
All tests pass except the one that uses both out-of-core graph and multiple compute threads.
The error is the following:
https://gist.github.com/4429650


  was:
~cmartella pointed out this issue: when using multithreaded computation in conjunction with
out-of-core graph, we incur in a race condition. The compute threads share the same DiskBackedPartitionStore,
whose getPartition() method is not meant to be thread-safe. When two threads request two out-of-core
partitions concurrently, they both try to load it to the same slot.
The result is that we can lose the reference to one of the two partitions (which will not
be written back to disk) and we can incur in a NullPointerException when both threads are
trying to offload the currently loaded partition to disk.

I ran this test to confirm the issue:
https://gist.github.com/4429628
All tests pass except the one that uses both out-of-core graph and multiple compute threads.
The error is the following:
https://gist.github.com/4429650


    
> Multithreading breaks out-of-core graph
> ---------------------------------------
>
>                 Key: GIRAPH-462
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-462
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Alessandro Presta
>            Priority: Critical
>
> [~cmartella] pointed out this issue: when using multithreaded computation in conjunction
with out-of-core graph, we incur in a race condition. The compute threads share the same DiskBackedPartitionStore,
whose getPartition() method is not meant to be thread-safe. When two threads request two out-of-core
partitions concurrently, they both try to load it to the same slot.
> The result is that we can lose the reference to one of the two partitions (which will
not be written back to disk) and we can incur in a NullPointerException when both threads
are trying to offload the currently loaded partition to disk.
> I ran this test to confirm the issue:
> https://gist.github.com/4429628
> All tests pass except the one that uses both out-of-core graph and multiple compute threads.
> The error is the following:
> https://gist.github.com/4429650

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message