hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-443) SSSP doesn't works with multi-tasks
Date Mon, 12 Dec 2011 06:18:31 GMT

    [ https://issues.apache.org/jira/browse/HAMA-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167371#comment-13167371
] 

Edward J. Yoon commented on HAMA-443:
-------------------------------------

I met some troubles when testing sssp with 10 million ~ 1 billion vertices inputs.

1. Task number 10 received vertexID 3831904.

{code}
# tail -100 core/logs/tasklogs/1/attempt_201112121510_0001_000010_0.log

11/12/12 14:47:58 INFO examples.ShortestPaths: >> 3831904
11/12/12 14:47:58 WARN bsp.GroomServer: Error running child
java.lang.ArrayIndexOutOfBoundsException: -1095543
{code}

2. Fail to communicate with BSPMaster for reporting

{code}
2011-12-12 10:40:15,042 INFO org.apache.hama.bsp.GroomServer: Launch 6
tasks.2011-12-12 10:40:16,298 INFO org.apache.hama.bsp.GroomServer:
Task 'attempt_201112121035_0002_000012_0' has started.2011-12-12
10:40:16,521 INFO org.apache.hama.bsp.GroomServer: Task
'attempt_201112121035_0002_000013_0' has started.2011-12-12
10:40:16,749 INFO org.apache.hama.bsp.GroomServer: Task
'attempt_201112121035_0002_000014_0' has started.2011-12-12
10:40:17,097 INFO org.apache.hama.bsp.GroomServer: Task
'attempt_201112121035_0002_000015_0' has started.2011-12-12
10:40:17,381 INFO org.apache.hama.bsp.GroomServer: Task
'attempt_201112121035_0002_000016_0' has started.2011-12-12
10:40:17,748 INFO org.apache.hama.bsp.GroomServer: Task
'attempt_201112121035_0002_000017_0' has started.2011-12-12
10:47:36,064 ERROR org.apache.hama.bsp.GroomServer: Fail to
communicate with BSPMaster for reporting.java.io.IOException: Call to
hnode1/10.33.2.101:40000 failed on local exception:
java.io.IOException: Connection reset by peer        at
org.apache.hadoop.ipc.Client.wrapException(Client.java:775)        at
org.apache.hadoop.ipc.Client.call(Client.java:743)        at
org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)        at
$Proxy4.report(Unknown Source)        at
org.apache.hama.bsp.GroomServer.doReport(GroomServer.java:482)
at org.apache.hama.bsp.GroomServer.offerService(GroomServer.java:408)
      at org.apache.hama.bsp.GroomServer.run(GroomServer.java:644)
   at java.lang.Thread.run(Thread.java:619)Caused by:
java.io.IOException: Connection reset by peer        at
sun.nio.ch.FileDispatcher.read0(Native Method)        at
sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)        at
sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:237)        at
sun.nio.ch.IOUtil.read(IOUtil.java:210)        at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
       at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
       at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
       at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
       at java.io.FilterInputStream.read(FilterInputStream.java:116)
     at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:276)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
       at java.io.DataInputStream.readInt(DataInputStream.java:370)
    at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
       at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
{code}



                
> SSSP doesn't works with multi-tasks
> -----------------------------------
>
>                 Key: HAMA-443
>                 URL: https://issues.apache.org/jira/browse/HAMA-443
>             Project: Hama
>          Issue Type: Bug
>          Components: examples
>    Affects Versions: 0.3.0
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 0.4.0
>
>         Attachments: HAMA-443.patch, HAMA-443_1.patch, HAMA-443_v2.patch, HAMA-443_v3.patch,
HAMA-443_v4.patch, HAMA-443_v5.patch, mapToList.patch, patch.txt, refactor.patch, test.patch,
test2.patch
>
>
> {code}
> root@Cnode1:/usr/local/src/hama-trunk# core/bin/hama jar examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar
sssp Klewno xx /user/root/edward/sssp-adjacencylist.txt
> Single Source Shortest Path Example:
> <Startvertex name> <optional: output path> <optional: path to own adjacency
list textfile!>
> Setting default start vertex to "Frankfurt"!
> Setting start vertex to Klewno!
> Using new output folder: xx
> 11/09/26 09:46:24 INFO graph.ShortestPaths: Starting data partitioning...
> 11/09/26 09:47:12 INFO graph.ShortestPaths: Finished!
> 11/09/26 09:47:12 INFO bsp.BSPJobClient: Running job: job_201109260929_0004
> 11/09/26 09:47:15 INFO bsp.BSPJobClient: Current supersteps number: 0
> 11/09/26 09:47:21 INFO bsp.BSPJobClient: Current supersteps number: 1
> 11/09/26 09:47:33 INFO bsp.BSPJobClient: The total number of supersteps: 1
> Job Finished in 21.553 seconds
> -------------------- RESULTS --------------------
> java.lang.NullPointerException
>         at org.apache.hama.examples.graph.ShortestPathsBase.printOutput(ShortestPathsBase.java:93)
>         at org.apache.hama.examples.graph.ShortestPaths.main(ShortestPaths.java:239)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>         at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>         at org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:37)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hama.util.RunJar.main(RunJar.java:145)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message