incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-443) SSSP doesn't works with multi-tasks
Date Fri, 11 Nov 2011 06:26:51 GMT

    [ https://issues.apache.org/jira/browse/HAMA-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148294#comment-13148294
] 

Thomas Jungblut commented on HAMA-443:
--------------------------------------

Why do I have to set this via the constructor when we have the method to set the class?

bq. What happens if splits size is bigger than cluster capacity?

Well, that is what I've meant by it is currently not deterministic.
Consider we provide the number of tasks = max task capacity of the cluster.
-> Partitioning runs and partitions to the number of tasks
-> then the filesplit sees that the files are already in the size of the number of tasks
and uses it as a filesplit.

That is what I have planned. But sometimes this is not going to work, especially when the
number of tasks is not set. Then the filesplit is totally doing no filesplit.

bq. With args, I received below errors 

Use the new file please.
http://code.google.com/p/hama-files/source/browse/trunk/HamaFiles/files/sssp-adjacencylist.seq
But anyways, the partitioning seems not to be stable at all. But it seeems a HDFS problem.

bq. As I asked before, what happens if tasks size is bigger than hard-coded 16 nodes?

I'm sorry. The tasks where no vertices are should do nothing. But they take slots.

bq. Let's just remove hard-coded data. And the usage should be printed only if args.length
< 3.

Yep, +1.

Sorry, but it is quite early in the morning.
                
> SSSP doesn't works with multi-tasks
> -----------------------------------
>
>                 Key: HAMA-443
>                 URL: https://issues.apache.org/jira/browse/HAMA-443
>             Project: Hama
>          Issue Type: Bug
>          Components: examples
>    Affects Versions: 0.3.0
>            Reporter: Edward J. Yoon
>            Assignee: Thomas Jungblut
>             Fix For: 0.4.0
>
>         Attachments: HAMA-443.patch, HAMA-443_1.patch, HAMA-443_v2.patch, HAMA-443_v3.patch,
patch.txt
>
>
> {code}
> root@Cnode1:/usr/local/src/hama-trunk# core/bin/hama jar examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar
sssp Klewno xx /user/root/edward/sssp-adjacencylist.txt
> Single Source Shortest Path Example:
> <Startvertex name> <optional: output path> <optional: path to own adjacency
list textfile!>
> Setting default start vertex to "Frankfurt"!
> Setting start vertex to Klewno!
> Using new output folder: xx
> 11/09/26 09:46:24 INFO graph.ShortestPaths: Starting data partitioning...
> 11/09/26 09:47:12 INFO graph.ShortestPaths: Finished!
> 11/09/26 09:47:12 INFO bsp.BSPJobClient: Running job: job_201109260929_0004
> 11/09/26 09:47:15 INFO bsp.BSPJobClient: Current supersteps number: 0
> 11/09/26 09:47:21 INFO bsp.BSPJobClient: Current supersteps number: 1
> 11/09/26 09:47:33 INFO bsp.BSPJobClient: The total number of supersteps: 1
> Job Finished in 21.553 seconds
> -------------------- RESULTS --------------------
> java.lang.NullPointerException
>         at org.apache.hama.examples.graph.ShortestPathsBase.printOutput(ShortestPathsBase.java:93)
>         at org.apache.hama.examples.graph.ShortestPaths.main(ShortestPaths.java:239)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>         at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>         at org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:37)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hama.util.RunJar.main(RunJar.java:145)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message