giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-473) InputSplitPathOrganizer should be aware of multiple threads
Date Thu, 10 Jan 2013 19:46:12 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13549979#comment-13549979
] 

Eli Reisman commented on GIRAPH-473:
------------------------------------

Just looked this over. you're right on and just checking for "good" is fine IMHO.

I am pretty sure with the new threadId it's just mixing the last two up in the hashing. Since
these will now be the new "long term" reliable hash results, I'd almost switch "bad" for "ugly"
in the test code and leave the check on all 3, maybe with a comment that results could change
if the hashing scheme changes again in the future?

When I originally wrote this, there was at one point a subtle (accidental) change to the hashing
scheme showed up due to the sensitivity of the test that just checking for "good" would not
have caught. The details are escaping me at the moment. If this is too sensitive an approach,
I think you're safe just checking for "good" at index 0 in the result list.


                
> InputSplitPathOrganizer should be aware of multiple threads
> -----------------------------------------------------------
>
>                 Key: GIRAPH-473
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-473
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Alessandro Presta
>            Assignee: Alessandro Presta
>            Priority: Minor
>         Attachments: GIRAPH-473.patch
>
>
> The starting offset for iterating over input splits is obtained by hashing the worker's
hostname and port. When multiple input threads are used, they get the same offset.
> We should hash the thread id as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message