incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HAMA-413) Remove limitation on the number of tasks
Date Tue, 23 Aug 2011 12:55:29 GMT

     [ https://issues.apache.org/jira/browse/HAMA-413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Edward J. Yoon updated HAMA-413:
--------------------------------

    Attachment: HAMA-413_v03.patch

This patch is very dirty but runs well.

Tested Pi and Serialized Printing on physical 2 nodes, 2/6 tasks.

{code}
11/08/23 21:47:48 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir: hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_dxqugl
11/08/23 21:47:49 INFO bsp.BSPJobClient: Running job: job_201108232147_0001
11/08/23 21:47:52 INFO bsp.BSPJobClient: Current supersteps number: 0
11/08/23 21:47:58 INFO bsp.BSPJobClient: Current supersteps number: 2
11/08/23 21:47:58 INFO bsp.BSPJobClient: The total number of supersteps: 2
Each task printed the "Hello World" as below:
Tue Aug 23 21:47:53 KST 2011: Hello BSP from 1 of 2: slave.udanax.org:61000
Tue Aug 23 21:49:27 KST 2011: Hello BSP from 2 of 2: tweetple.com:61000
edward@slave:~/workspace/hama-trunk$ core/bin/hama jar ./examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar
pi
11/08/23 21:48:07 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir: hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_vmkvwb
11/08/23 21:48:09 INFO bsp.BSPJobClient: Running job: job_201108232147_0002
11/08/23 21:48:12 INFO bsp.BSPJobClient: Current supersteps number: 0
11/08/23 21:48:18 INFO bsp.BSPJobClient: Current supersteps number: 1
11/08/23 21:48:18 INFO bsp.BSPJobClient: The total number of supersteps: 1
Estimated value of PI is 3.1370000000000005

----

2011-08-23 21:48:12,515 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000001_0
11/08/23 21:48:12 INFO bsp.GroomServer: >>>>>> [slave.udanax.org:61000,
slave.udanax.org:61001, tweetple.com:61000, tweetple.com:61001, tweetple.com:61002, tweetple.com:61003]
2011-08-23 21:48:12,743 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000001_0
11/08/23 21:48:12 DEBUG bsp.BSPPeer: Local send bytes (3.1292)
2011-08-23 21:48:12,816 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000001_0
11/08/23 21:48:12 DEBUG bsp.BSPPeer: [slave.udanax.org:61000] enter the enterbarrier: 0
2011-08-23 21:48:12,824 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000003_0
11/08/23 21:48:12 DEBUG bsp.BSPPeer: Send bytes (3.1304) to slave.udanax.org:61000
2011-08-23 21:48:12,824 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000003_0
11/08/23 21:48:12 DEBUG bsp.BSPPeer: [slave.udanax.org:61001] enter the enterbarrier: 0
2011-08-23 21:48:16,525 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000003_0
11/08/23 21:48:16 INFO ipc.Server: Stopping server on 61001
2011-08-23 21:48:16,525 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000003_0
11/08/23 21:48:16 INFO ipc.Server: IPC Server handler 0 on 61001: exiting
2011-08-23 21:48:16,525 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000003_0
11/08/23 21:48:16 INFO ipc.Server: Stopping IPC Server listener on 61001
2011-08-23 21:48:16,526 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000003_0
11/08/23 21:48:16 INFO ipc.Server: Stopping IPC Server Responder
2011-08-23 21:48:17,072 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000001_0
11/08/23 21:48:17 INFO ipc.Server: Stopping server on 61000
2011-08-23 21:48:17,072 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000001_0
11/08/23 21:48:17 INFO ipc.Server: Stopping IPC Server listener on 61000
2011-08-23 21:48:17,072 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000001_0
11/08/23 21:48:17 INFO ipc.Server: IPC Server handler 0 on 61000: exiting
2011-08-23 21:48:17,072 INFO org.apache.hama.bsp.TaskRunner: attempt_201108232147_0002_000001_0
11/08/23 21:48:17 INFO ipc.Server: Stopping IPC Server Responder

{code}

> Remove limitation on the number of tasks
> ----------------------------------------
>
>                 Key: HAMA-413
>                 URL: https://issues.apache.org/jira/browse/HAMA-413
>             Project: Hama
>          Issue Type: Sub-task
>          Components: bsp
>    Affects Versions: 0.3.0
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 0.4.0
>
>         Attachments: HAMA-413_v01.patch, HAMA-413_v02.patch, HAMA-413_v03.patch
>
>
> By HAMA-410 patch, BSPPeer object will be constructed at child process. Now we can just
remove limitation on the number of tasks.
> Here's TODO list:
> 1. The number of tasks per groom should be configurable e.g., 'bsp.local.tasks.maximum'.
> 2. The 'totalTaskCapacity' should be calculated at BSPMaster.getClusterStatus().
> 3. When scheduling tasks, consider how to allocate them.
> 4. Each BSPPeer should know all created peers of Hama cluster by job. It can be listed
based on actions of GroomServer.
> 5. In examples, 'cluster.getGroomServers()' can be changed to 'cluster.getMaxTasks()'.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message