incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-443) SSSP doesn't works with multi-tasks
Date Thu, 24 Nov 2011 15:13:39 GMT

    [ https://issues.apache.org/jira/browse/HAMA-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156750#comment-13156750
] 

Thomas Jungblut commented on HAMA-443:
--------------------------------------

That is normal, in BSP the synchronizations are really expensive because they are global and
driven by the slowest node.

But over all, by looking at the algorithm, there is much more communication than computation.

Have you seen this?
http://www.slideshare.net/chodakowski/processing-graphrelational-data-with-mapreduce-and-bulk-synchronous-parallel

I've also posted it on twitter, maybe we can get a lot more of it by using an advanced partitioning.
("Shimmy trick" in the slides, this applies for M/R)
                
> SSSP doesn't works with multi-tasks
> -----------------------------------
>
>                 Key: HAMA-443
>                 URL: https://issues.apache.org/jira/browse/HAMA-443
>             Project: Hama
>          Issue Type: Bug
>          Components: examples
>    Affects Versions: 0.3.0
>            Reporter: Edward J. Yoon
>            Assignee: Thomas Jungblut
>             Fix For: 0.4.0
>
>         Attachments: HAMA-443.patch, HAMA-443_1.patch, HAMA-443_v2.patch, HAMA-443_v3.patch,
HAMA-443_v4.patch, HAMA-443_v5.patch, mapToList.patch, patch.txt, refactor.patch
>
>
> {code}
> root@Cnode1:/usr/local/src/hama-trunk# core/bin/hama jar examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar
sssp Klewno xx /user/root/edward/sssp-adjacencylist.txt
> Single Source Shortest Path Example:
> <Startvertex name> <optional: output path> <optional: path to own adjacency
list textfile!>
> Setting default start vertex to "Frankfurt"!
> Setting start vertex to Klewno!
> Using new output folder: xx
> 11/09/26 09:46:24 INFO graph.ShortestPaths: Starting data partitioning...
> 11/09/26 09:47:12 INFO graph.ShortestPaths: Finished!
> 11/09/26 09:47:12 INFO bsp.BSPJobClient: Running job: job_201109260929_0004
> 11/09/26 09:47:15 INFO bsp.BSPJobClient: Current supersteps number: 0
> 11/09/26 09:47:21 INFO bsp.BSPJobClient: Current supersteps number: 1
> 11/09/26 09:47:33 INFO bsp.BSPJobClient: The total number of supersteps: 1
> Job Finished in 21.553 seconds
> -------------------- RESULTS --------------------
> java.lang.NullPointerException
>         at org.apache.hama.examples.graph.ShortestPathsBase.printOutput(ShortestPathsBase.java:93)
>         at org.apache.hama.examples.graph.ShortestPaths.main(ShortestPaths.java:239)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>         at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>         at org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:37)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hama.util.RunJar.main(RunJar.java:145)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message