incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HAMA-154) [Performance]Combine multi-mapreduce jobs into a single mapreduce job while computing such as 'C=alpha*B + A'
Date Wed, 04 Mar 2009 09:11:56 GMT

    [ https://issues.apache.org/jira/browse/HAMA-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678648#action_12678648
] 

Edward J. Yoon commented on HAMA-154:
-------------------------------------

Full log:

[d8g053:/root/hama-trunk]# bin/hama examples add fiveA fiveB
09/03/04 17:35:15 INFO hama.AbstractMatrix: Initializing the matrix storage.
09/03/04 17:35:21 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randnomnp
09/03/04 17:35:21 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
09/03/04 17:35:21 WARN mapred.JobClient: Use genericOptions for the option -libjars
09/03/04 17:35:21 WARN mapred.JobClient: No job jar file set.  User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 0->d8g053.nhncorp.com:,000000000001394
09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001394,000000000003146
09/03/04 17:35:21 INFO mapred.TableInputFormatBase: split: 2->d8g054.nhncorp.com:000000000003146,
09/03/04 17:35:21 INFO mapred.JobClient: Running job: job_200902131649_0036
09/03/04 17:35:22 INFO mapred.JobClient:  map 0% reduce 0%
09/03/04 17:37:44 INFO mapred.JobClient:  map 33% reduce 0%
09/03/04 17:37:51 INFO mapred.JobClient:  map 33% reduce 3%
09/03/04 17:37:54 INFO mapred.JobClient:  map 33% reduce 7%
09/03/04 17:38:00 INFO mapred.JobClient:  map 33% reduce 11%
09/03/04 17:38:26 INFO mapred.JobClient:  map 66% reduce 11%
09/03/04 17:38:32 INFO mapred.JobClient:  map 66% reduce 14%
09/03/04 17:38:33 INFO mapred.JobClient:  map 100% reduce 14%
09/03/04 17:38:40 INFO mapred.JobClient:  map 100% reduce 18%
09/03/04 17:38:43 INFO mapred.JobClient:  map 100% reduce 37%
09/03/04 17:38:48 INFO mapred.JobClient:  map 100% reduce 53%
09/03/04 17:38:50 INFO mapred.JobClient:  map 100% reduce 68%
09/03/04 17:38:55 INFO mapred.JobClient:  map 100% reduce 69%
09/03/04 17:39:00 INFO mapred.JobClient:  map 100% reduce 70%
09/03/04 17:39:08 INFO mapred.JobClient:  map 100% reduce 71%
09/03/04 17:39:13 INFO mapred.JobClient:  map 100% reduce 72%
09/03/04 17:39:18 INFO mapred.JobClient:  map 100% reduce 73%
09/03/04 17:39:23 INFO mapred.JobClient:  map 100% reduce 74%
09/03/04 17:39:28 INFO mapred.JobClient:  map 100% reduce 75%
09/03/04 17:39:35 INFO mapred.JobClient:  map 100% reduce 76%
09/03/04 17:39:40 INFO mapred.JobClient:  map 100% reduce 77%
09/03/04 17:39:53 INFO mapred.JobClient:  map 100% reduce 78%
09/03/04 17:39:58 INFO mapred.JobClient:  map 100% reduce 79%
09/03/04 17:40:03 INFO mapred.JobClient:  map 100% reduce 80%
09/03/04 17:40:08 INFO mapred.JobClient:  map 100% reduce 81%
09/03/04 17:40:13 INFO mapred.JobClient:  map 100% reduce 82%
09/03/04 17:40:20 INFO mapred.JobClient:  map 100% reduce 83%
09/03/04 17:40:27 INFO mapred.JobClient:  map 100% reduce 84%
09/03/04 17:40:32 INFO mapred.JobClient:  map 100% reduce 85%
09/03/04 17:40:38 INFO mapred.JobClient:  map 100% reduce 86%
09/03/04 17:40:43 INFO mapred.JobClient:  map 100% reduce 87%
09/03/04 17:40:57 INFO mapred.JobClient:  map 100% reduce 88%
09/03/04 17:41:02 INFO mapred.JobClient:  map 100% reduce 89%
09/03/04 17:41:08 INFO mapred.JobClient:  map 100% reduce 90%
09/03/04 17:41:13 INFO mapred.JobClient:  map 100% reduce 91%
09/03/04 17:41:19 INFO mapred.JobClient:  map 100% reduce 92%
09/03/04 17:41:25 INFO mapred.JobClient:  map 100% reduce 93%
09/03/04 17:41:33 INFO mapred.JobClient:  map 100% reduce 94%
09/03/04 17:41:38 INFO mapred.JobClient:  map 100% reduce 95%
09/03/04 17:41:43 INFO mapred.JobClient:  map 100% reduce 96%
09/03/04 17:41:50 INFO mapred.JobClient:  map 100% reduce 97%
09/03/04 17:41:55 INFO mapred.JobClient:  map 100% reduce 98%
09/03/04 17:42:02 INFO mapred.JobClient:  map 100% reduce 99%
09/03/04 17:42:05 INFO mapred.JobClient:  map 100% reduce 100%
09/03/04 17:42:10 INFO mapred.JobClient: Job complete: job_200902131649_0036
09/03/04 17:42:10 INFO mapred.JobClient: Counters: 15
09/03/04 17:42:10 INFO mapred.JobClient:   File Systems
09/03/04 17:42:10 INFO mapred.JobClient:     Local bytes read=1239653931
09/03/04 17:42:10 INFO mapred.JobClient:     Local bytes written=1859371053
09/03/04 17:42:10 INFO mapred.JobClient:   Job Counters 
09/03/04 17:42:10 INFO mapred.JobClient:     Launched reduce tasks=3
09/03/04 17:42:10 INFO mapred.JobClient:     Rack-local map tasks=2
09/03/04 17:42:10 INFO mapred.JobClient:     Launched map tasks=5
09/03/04 17:42:10 INFO mapred.JobClient:     Data-local map tasks=3
09/03/04 17:42:10 INFO mapred.JobClient:   Map-Reduce Framework
09/03/04 17:42:10 INFO mapred.JobClient:     Reduce input groups=5001
09/03/04 17:42:10 INFO mapred.JobClient:     Combine output records=0
09/03/04 17:42:10 INFO mapred.JobClient:     Map input records=5001
09/03/04 17:42:10 INFO mapred.JobClient:     Reduce output records=5001
09/03/04 17:42:10 INFO mapred.JobClient:     Map output bytes=619765182
09/03/04 17:42:10 INFO mapred.JobClient:     Map input bytes=0
09/03/04 17:42:10 INFO mapred.JobClient:     Combine input records=0
09/03/04 17:42:10 INFO mapred.JobClient:     Map output records=5001
09/03/04 17:42:10 INFO mapred.JobClient:     Reduce input records=5001
09/03/04 17:42:13 INFO hama.AbstractMatrix: Initializing the matrix storage.
09/03/04 17:42:17 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randeeiww
09/03/04 17:42:17 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
09/03/04 17:42:17 WARN mapred.JobClient: Use genericOptions for the option -libjars
09/03/04 17:42:18 WARN mapred.JobClient: No job jar file set.  User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 0->d8g054.nhncorp.com:,000000000001367
09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001367,000000000003424
09/03/04 17:42:18 INFO mapred.HTableInputFormatBase: split: 2->d8g055.nhncorp.com:000000000003424,
09/03/04 17:42:18 INFO mapred.JobClient: Running job: job_200902131649_0037
09/03/04 17:42:19 INFO mapred.JobClient:  map 0% reduce 0%
09/03/04 17:42:43 INFO mapred.JobClient:  map 1% reduce 0%
09/03/04 17:42:50 INFO mapred.JobClient:  map 2% reduce 0%
09/03/04 17:42:53 INFO mapred.JobClient:  map 3% reduce 0%
09/03/04 17:42:58 INFO mapred.JobClient:  map 5% reduce 0%
09/03/04 17:43:04 INFO mapred.JobClient:  map 6% reduce 0%
09/03/04 17:43:08 INFO mapred.JobClient:  map 7% reduce 0%
09/03/04 17:43:13 INFO mapred.JobClient:  map 8% reduce 0%
09/03/04 17:43:18 INFO mapred.JobClient:  map 9% reduce 0%
09/03/04 17:43:23 INFO mapred.JobClient:  map 10% reduce 0%
09/03/04 17:43:24 INFO mapred.JobClient:  map 11% reduce 0%
09/03/04 17:43:28 INFO mapred.JobClient:  map 12% reduce 0%
09/03/04 17:43:33 INFO mapred.JobClient:  map 13% reduce 0%
09/03/04 17:43:38 INFO mapred.JobClient:  map 14% reduce 0%
09/03/04 17:43:40 INFO mapred.JobClient:  map 15% reduce 0%
09/03/04 17:43:44 INFO mapred.JobClient:  map 16% reduce 0%
09/03/04 17:43:50 INFO mapred.JobClient:  map 17% reduce 0%
09/03/04 17:43:53 INFO mapred.JobClient:  map 18% reduce 0%
09/03/04 17:43:58 INFO mapred.JobClient:  map 19% reduce 0%
09/03/04 17:44:03 INFO mapred.JobClient:  map 20% reduce 0%
09/03/04 17:44:13 INFO mapred.JobClient:  map 21% reduce 0%
09/03/04 17:44:18 INFO mapred.JobClient:  map 22% reduce 0%
09/03/04 17:44:23 INFO mapred.JobClient:  map 23% reduce 0%
09/03/04 17:44:33 INFO mapred.JobClient:  map 24% reduce 0%
09/03/04 17:44:35 INFO mapred.JobClient:  map 25% reduce 0%
09/03/04 17:44:38 INFO mapred.JobClient:  map 26% reduce 0%
09/03/04 17:44:45 INFO mapred.JobClient:  map 27% reduce 0%
09/03/04 17:44:48 INFO mapred.JobClient:  map 28% reduce 0%
09/03/04 17:44:53 INFO mapred.JobClient:  map 29% reduce 0%
09/03/04 17:44:58 INFO mapred.JobClient:  map 30% reduce 0%
09/03/04 17:45:03 INFO mapred.JobClient:  map 31% reduce 0%
09/03/04 17:45:04 INFO mapred.JobClient:  map 32% reduce 0%
09/03/04 17:45:08 INFO mapred.JobClient:  map 33% reduce 0%
09/03/04 17:45:14 INFO mapred.JobClient:  map 34% reduce 0%
09/03/04 17:45:15 INFO mapred.JobClient:  map 35% reduce 0%
09/03/04 17:45:20 INFO mapred.JobClient:  map 36% reduce 0%
09/03/04 17:45:24 INFO mapred.JobClient:  map 37% reduce 0%
09/03/04 17:45:30 INFO mapred.JobClient:  map 38% reduce 0%
09/03/04 17:45:39 INFO mapred.JobClient:  map 39% reduce 0%
09/03/04 17:45:44 INFO mapred.JobClient:  map 40% reduce 0%
09/03/04 17:45:49 INFO mapred.JobClient:  map 41% reduce 0%
09/03/04 17:45:54 INFO mapred.JobClient:  map 42% reduce 0%
09/03/04 17:45:59 INFO mapred.JobClient:  map 43% reduce 0%
09/03/04 17:46:08 INFO mapred.JobClient:  map 45% reduce 0%
09/03/04 17:46:25 INFO mapred.JobClient:  map 45% reduce 3%
09/03/04 17:46:26 INFO mapred.JobClient:  map 45% reduce 7%
09/03/04 17:46:28 INFO mapred.JobClient:  map 45% reduce 11%
09/03/04 17:46:55 INFO mapred.JobClient:  map 46% reduce 11%
09/03/04 17:47:05 INFO mapred.JobClient:  map 47% reduce 11%
09/03/04 17:47:21 INFO mapred.JobClient:  map 48% reduce 11%
09/03/04 17:47:30 INFO mapred.JobClient:  map 49% reduce 11%
09/03/04 17:47:35 INFO mapred.JobClient:  map 50% reduce 11%
09/03/04 17:47:50 INFO mapred.JobClient:  map 51% reduce 11%
09/03/04 17:48:00 INFO mapred.JobClient:  map 52% reduce 11%
09/03/04 17:48:05 INFO mapred.JobClient:  map 53% reduce 11%
09/03/04 17:48:15 INFO mapred.JobClient:  map 54% reduce 11%
09/03/04 17:48:30 INFO mapred.JobClient:  map 55% reduce 11%
09/03/04 17:48:35 INFO mapred.JobClient:  map 56% reduce 11%
09/03/04 17:48:45 INFO mapred.JobClient:  map 57% reduce 11%
09/03/04 17:48:55 INFO mapred.JobClient:  map 58% reduce 11%
09/03/04 17:49:40 INFO mapred.JobClient:  map 59% reduce 11%
09/03/04 17:49:50 INFO mapred.JobClient:  map 60% reduce 11%
09/03/04 17:50:00 INFO mapred.JobClient:  map 61% reduce 11%
09/03/04 17:50:10 INFO mapred.JobClient:  map 62% reduce 11%
09/03/04 17:50:21 INFO mapred.JobClient:  map 63% reduce 11%
09/03/04 17:50:31 INFO mapred.JobClient:  map 64% reduce 11%
09/03/04 17:50:41 INFO mapred.JobClient:  map 65% reduce 11%
09/03/04 17:50:51 INFO mapred.JobClient:  map 66% reduce 11%
09/03/04 17:51:04 INFO mapred.JobClient:  map 66% reduce 14%
09/03/04 17:51:09 INFO mapred.JobClient:  map 66% reduce 18%
09/03/04 17:51:12 INFO mapred.JobClient:  map 66% reduce 22%
09/03/04 17:51:47 INFO mapred.JobClient:  map 100% reduce 22%
09/03/04 17:51:59 INFO mapred.JobClient:  map 100% reduce 52%
09/03/04 17:52:02 INFO mapred.JobClient:  map 100% reduce 67%
09/03/04 17:52:04 INFO mapred.JobClient:  map 100% reduce 68%
09/03/04 17:52:09 INFO mapred.JobClient:  map 100% reduce 69%
09/03/04 17:52:14 INFO mapred.JobClient:  map 100% reduce 70%
09/03/04 17:52:19 INFO mapred.JobClient:  map 100% reduce 71%
09/03/04 17:52:27 INFO mapred.JobClient:  map 100% reduce 72%
09/03/04 17:52:32 INFO mapred.JobClient:  map 100% reduce 73%
09/03/04 17:52:37 INFO mapred.JobClient:  map 100% reduce 74%
09/03/04 17:52:44 INFO mapred.JobClient:  map 100% reduce 75%
09/03/04 17:52:49 INFO mapred.JobClient:  map 100% reduce 76%
09/03/04 17:52:57 INFO mapred.JobClient:  map 100% reduce 77%
09/03/04 17:53:12 INFO mapred.JobClient:  map 100% reduce 78%
09/03/04 17:53:14 INFO mapred.JobClient:  map 100% reduce 79%
09/03/04 17:53:19 INFO mapred.JobClient:  map 100% reduce 80%
09/03/04 17:53:27 INFO mapred.JobClient:  map 100% reduce 81%
09/03/04 17:53:32 INFO mapred.JobClient:  map 100% reduce 82%
09/03/04 17:53:39 INFO mapred.JobClient:  map 100% reduce 83%
09/03/04 17:53:44 INFO mapred.JobClient:  map 100% reduce 84%
09/03/04 17:53:49 INFO mapred.JobClient:  map 100% reduce 85%
09/03/04 17:53:57 INFO mapred.JobClient:  map 100% reduce 86%
09/03/04 17:54:02 INFO mapred.JobClient:  map 100% reduce 87%
09/03/04 17:54:19 INFO mapred.JobClient:  map 100% reduce 88%
09/03/04 17:54:28 INFO mapred.JobClient:  map 100% reduce 89%
09/03/04 17:54:33 INFO mapred.JobClient:  map 100% reduce 90%
09/03/04 17:54:39 INFO mapred.JobClient:  map 100% reduce 91%
09/03/04 17:54:44 INFO mapred.JobClient:  map 100% reduce 92%
09/03/04 17:54:49 INFO mapred.JobClient:  map 100% reduce 93%
09/03/04 17:54:58 INFO mapred.JobClient:  map 100% reduce 94%
09/03/04 17:55:04 INFO mapred.JobClient:  map 100% reduce 95%
09/03/04 17:55:09 INFO mapred.JobClient:  map 100% reduce 96%
09/03/04 17:55:14 INFO mapred.JobClient:  map 100% reduce 97%
09/03/04 17:55:19 INFO mapred.JobClient:  map 100% reduce 98%
09/03/04 17:55:28 INFO mapred.JobClient:  map 100% reduce 99%
09/03/04 17:55:31 INFO mapred.JobClient:  map 100% reduce 100%
09/03/04 17:55:36 INFO mapred.JobClient: Job complete: job_200902131649_0037
09/03/04 17:55:36 INFO mapred.JobClient: Counters: 15
09/03/04 17:55:36 INFO mapred.JobClient:   File Systems
09/03/04 17:55:36 INFO mapred.JobClient:     Local bytes read=1350551934
09/03/04 17:55:36 INFO mapred.JobClient:     Local bytes written=2025705516
09/03/04 17:55:36 INFO mapred.JobClient:   Job Counters 
09/03/04 17:55:36 INFO mapred.JobClient:     Launched reduce tasks=3
09/03/04 17:55:36 INFO mapred.JobClient:     Rack-local map tasks=1
09/03/04 17:55:36 INFO mapred.JobClient:     Launched map tasks=4
09/03/04 17:55:36 INFO mapred.JobClient:     Data-local map tasks=3
09/03/04 17:55:36 INFO mapred.JobClient:   Map-Reduce Framework
09/03/04 17:55:36 INFO mapred.JobClient:     Reduce input groups=5000
09/03/04 17:55:36 INFO mapred.JobClient:     Combine output records=0
09/03/04 17:55:36 INFO mapred.JobClient:     Map input records=5000
09/03/04 17:55:36 INFO mapred.JobClient:     Reduce output records=5000
09/03/04 17:55:36 INFO mapred.JobClient:     Map output bytes=675210000
09/03/04 17:55:36 INFO mapred.JobClient:     Map input bytes=0
09/03/04 17:55:36 INFO mapred.JobClient:     Combine input records=0
09/03/04 17:55:36 INFO mapred.JobClient:     Map output records=5000
09/03/04 17:55:36 INFO mapred.JobClient:     Reduce input records=5000

---------------------------------------

[d8g053:/root/hama-trunk]# bin/hama examples add fiveA fiveB
09/03/04 17:59:06 INFO hama.AbstractMatrix: Initializing the matrix storage.
09/03/04 17:59:12 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randxfeud
09/03/04 17:59:13 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
09/03/04 17:59:13 WARN mapred.JobClient: Use genericOptions for the option -libjars
09/03/04 17:59:13 WARN mapred.JobClient: No job jar file set.  User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 0->d8g054.nhncorp.com:,000000000001367
09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 1->d8g053.nhncorp.com:000000000001367,000000000003424
09/03/04 17:59:13 INFO mapred.HTableInputFormatBase: split: 2->d8g055.nhncorp.com:000000000003424,
09/03/04 17:59:13 INFO mapred.JobClient: Running job: job_200902131649_0038
09/03/04 17:59:14 INFO mapred.JobClient:  map 0% reduce 0%
09/03/04 17:59:41 INFO mapred.JobClient:  map 1% reduce 0%
09/03/04 17:59:44 INFO mapred.JobClient:  map 2% reduce 0%
09/03/04 17:59:46 INFO mapred.JobClient:  map 3% reduce 0%
09/03/04 17:59:54 INFO mapred.JobClient:  map 4% reduce 0%
09/03/04 17:59:56 INFO mapred.JobClient:  map 5% reduce 0%
09/03/04 18:00:01 INFO mapred.JobClient:  map 6% reduce 0%
09/03/04 18:00:04 INFO mapred.JobClient:  map 7% reduce 0%
09/03/04 18:00:09 INFO mapred.JobClient:  map 8% reduce 0%
09/03/04 18:00:11 INFO mapred.JobClient:  map 9% reduce 0%
09/03/04 18:00:16 INFO mapred.JobClient:  map 10% reduce 0%
09/03/04 18:00:19 INFO mapred.JobClient:  map 11% reduce 0%
09/03/04 18:00:26 INFO mapred.JobClient:  map 12% reduce 0%
09/03/04 18:00:29 INFO mapred.JobClient:  map 13% reduce 0%
09/03/04 18:00:31 INFO mapred.JobClient:  map 14% reduce 0%
09/03/04 18:00:34 INFO mapred.JobClient:  map 15% reduce 0%
09/03/04 18:00:41 INFO mapred.JobClient:  map 16% reduce 0%
09/03/04 18:00:44 INFO mapred.JobClient:  map 17% reduce 0%
09/03/04 18:00:46 INFO mapred.JobClient:  map 18% reduce 0%
09/03/04 18:00:51 INFO mapred.JobClient:  map 19% reduce 0%
09/03/04 18:00:56 INFO mapred.JobClient:  map 20% reduce 0%
09/03/04 18:00:59 INFO mapred.JobClient:  map 21% reduce 0%
09/03/04 18:01:04 INFO mapred.JobClient:  map 22% reduce 0%
09/03/04 18:01:06 INFO mapred.JobClient:  map 23% reduce 0%
09/03/04 18:01:11 INFO mapred.JobClient:  map 24% reduce 0%
09/03/04 18:01:16 INFO mapred.JobClient:  map 25% reduce 0%
09/03/04 18:01:18 INFO mapred.JobClient:  map 26% reduce 0%
09/03/04 18:01:24 INFO mapred.JobClient:  map 27% reduce 0%
09/03/04 18:01:27 INFO mapred.JobClient:  map 28% reduce 0%
09/03/04 18:01:32 INFO mapred.JobClient:  map 29% reduce 0%
09/03/04 18:01:34 INFO mapred.JobClient:  map 30% reduce 0%
09/03/04 18:01:42 INFO mapred.JobClient:  map 31% reduce 0%
09/03/04 18:01:44 INFO mapred.JobClient:  map 32% reduce 0%
09/03/04 18:01:47 INFO mapred.JobClient:  map 33% reduce 0%
09/03/04 18:01:52 INFO mapred.JobClient:  map 34% reduce 0%
09/03/04 18:01:57 INFO mapred.JobClient:  map 35% reduce 0%
09/03/04 18:01:59 INFO mapred.JobClient:  map 36% reduce 0%
09/03/04 18:02:02 INFO mapred.JobClient:  map 37% reduce 0%
09/03/04 18:02:07 INFO mapred.JobClient:  map 38% reduce 0%
09/03/04 18:02:12 INFO mapred.JobClient:  map 39% reduce 0%
09/03/04 18:02:14 INFO mapred.JobClient:  map 40% reduce 0%
09/03/04 18:02:17 INFO mapred.JobClient:  map 41% reduce 0%
09/03/04 18:02:22 INFO mapred.JobClient:  map 42% reduce 0%
09/03/04 18:02:27 INFO mapred.JobClient:  map 43% reduce 0%
09/03/04 18:02:29 INFO mapred.JobClient:  map 44% reduce 0%
09/03/04 18:02:32 INFO mapred.JobClient:  map 45% reduce 0%
09/03/04 18:02:37 INFO mapred.JobClient:  map 46% reduce 0%
09/03/04 18:02:42 INFO mapred.JobClient:  map 47% reduce 0%
09/03/04 18:02:47 INFO mapred.JobClient:  map 49% reduce 0%
09/03/04 18:02:52 INFO mapred.JobClient:  map 50% reduce 0%
09/03/04 18:02:57 INFO mapred.JobClient:  map 51% reduce 0%
09/03/04 18:02:59 INFO mapred.JobClient:  map 52% reduce 0%
09/03/04 18:03:02 INFO mapred.JobClient:  map 53% reduce 0%
09/03/04 18:03:07 INFO mapred.JobClient:  map 54% reduce 0%
09/03/04 18:03:11 INFO mapred.JobClient:  map 55% reduce 0%
09/03/04 18:03:19 INFO mapred.JobClient:  map 56% reduce 0%
09/03/04 18:03:20 INFO mapred.JobClient:  map 56% reduce 3%
09/03/04 18:03:21 INFO mapred.JobClient:  map 56% reduce 7%
09/03/04 18:03:24 INFO mapred.JobClient:  map 56% reduce 11%
09/03/04 18:03:29 INFO mapred.JobClient:  map 57% reduce 11%
09/03/04 18:03:34 INFO mapred.JobClient:  map 58% reduce 11%
09/03/04 18:03:44 INFO mapred.JobClient:  map 59% reduce 11%
09/03/04 18:03:50 INFO mapred.JobClient:  map 92% reduce 11%
09/03/04 18:03:54 INFO mapred.JobClient:  map 93% reduce 11%
09/03/04 18:04:01 INFO mapred.JobClient:  map 93% reduce 14%
09/03/04 18:04:04 INFO mapred.JobClient:  map 94% reduce 18%
09/03/04 18:04:05 INFO mapred.JobClient:  map 94% reduce 22%
09/03/04 18:04:14 INFO mapred.JobClient:  map 95% reduce 22%
09/03/04 18:04:19 INFO mapred.JobClient:  map 96% reduce 22%
09/03/04 18:04:29 INFO mapred.JobClient:  map 97% reduce 22%
09/03/04 18:04:34 INFO mapred.JobClient:  map 98% reduce 22%
09/03/04 18:04:44 INFO mapred.JobClient:  map 99% reduce 22%
09/03/04 18:04:53 INFO mapred.JobClient:  map 100% reduce 22%
09/03/04 18:05:02 INFO mapred.JobClient:  map 100% reduce 37%
09/03/04 18:05:03 INFO mapred.JobClient:  map 100% reduce 52%
09/03/04 18:05:05 INFO mapred.JobClient:  map 100% reduce 67%
09/03/04 18:05:06 INFO mapred.JobClient:  map 100% reduce 68%
09/03/04 18:05:13 INFO mapred.JobClient:  map 100% reduce 69%
09/03/04 18:05:18 INFO mapred.JobClient:  map 100% reduce 70%
09/03/04 18:05:23 INFO mapred.JobClient:  map 100% reduce 71%
09/03/04 18:05:30 INFO mapred.JobClient:  map 100% reduce 72%
09/03/04 18:05:35 INFO mapred.JobClient:  map 100% reduce 73%
09/03/04 18:05:41 INFO mapred.JobClient:  map 100% reduce 74%
09/03/04 18:05:48 INFO mapred.JobClient:  map 100% reduce 75%
09/03/04 18:05:51 INFO mapred.JobClient:  map 100% reduce 76%
09/03/04 18:05:58 INFO mapred.JobClient:  map 100% reduce 77%
09/03/04 18:06:05 INFO mapred.JobClient:  map 100% reduce 78%
09/03/04 18:06:10 INFO mapred.JobClient:  map 100% reduce 79%
09/03/04 18:06:17 INFO mapred.JobClient:  map 100% reduce 80%
09/03/04 18:06:21 INFO mapred.JobClient:  map 100% reduce 81%
09/03/04 18:06:36 INFO mapred.JobClient:  map 100% reduce 82%
09/03/04 18:06:43 INFO mapred.JobClient:  map 100% reduce 83%
09/03/04 18:06:50 INFO mapred.JobClient:  map 100% reduce 84%
09/03/04 18:06:55 INFO mapred.JobClient:  map 100% reduce 85%
09/03/04 18:07:01 INFO mapred.JobClient:  map 100% reduce 86%
09/03/04 18:07:07 INFO mapred.JobClient:  map 100% reduce 87%
09/03/04 18:07:14 INFO mapred.JobClient:  map 100% reduce 88%
09/03/04 18:07:20 INFO mapred.JobClient:  map 100% reduce 89%
09/03/04 18:07:27 INFO mapred.JobClient:  map 100% reduce 90%
09/03/04 18:07:34 INFO mapred.JobClient:  map 100% reduce 91%
09/03/04 18:07:39 INFO mapred.JobClient:  map 100% reduce 92%
09/03/04 18:07:45 INFO mapred.JobClient:  map 100% reduce 93%
09/03/04 18:07:51 INFO mapred.JobClient:  map 100% reduce 94%
09/03/04 18:07:58 INFO mapred.JobClient:  map 100% reduce 95%
09/03/04 18:08:10 INFO mapred.JobClient:  map 100% reduce 96%
09/03/04 18:08:16 INFO mapred.JobClient:  map 100% reduce 97%
09/03/04 18:08:21 INFO mapred.JobClient:  map 100% reduce 98%
09/03/04 18:08:27 INFO mapred.JobClient:  map 100% reduce 99%
09/03/04 18:08:32 INFO mapred.JobClient:  map 100% reduce 100%
09/03/04 18:08:38 INFO mapred.JobClient: Job complete: job_200902131649_0038
09/03/04 18:08:38 INFO mapred.JobClient: Counters: 15
09/03/04 18:08:38 INFO mapred.JobClient:   File Systems
09/03/04 18:08:38 INFO mapred.JobClient:     Local bytes read=1350551934
09/03/04 18:08:38 INFO mapred.JobClient:     Local bytes written=2025705516
09/03/04 18:08:38 INFO mapred.JobClient:   Job Counters 
09/03/04 18:08:38 INFO mapred.JobClient:     Launched reduce tasks=3
09/03/04 18:08:38 INFO mapred.JobClient:     Rack-local map tasks=1
09/03/04 18:08:38 INFO mapred.JobClient:     Launched map tasks=4
09/03/04 18:08:38 INFO mapred.JobClient:     Data-local map tasks=3
09/03/04 18:08:38 INFO mapred.JobClient:   Map-Reduce Framework
09/03/04 18:08:38 INFO mapred.JobClient:     Reduce input groups=5000
09/03/04 18:08:38 INFO mapred.JobClient:     Combine output records=0
09/03/04 18:08:38 INFO mapred.JobClient:     Map input records=5000
09/03/04 18:08:38 INFO mapred.JobClient:     Reduce output records=5000
09/03/04 18:08:38 INFO mapred.JobClient:     Map output bytes=675210000
09/03/04 18:08:38 INFO mapred.JobClient:     Map input bytes=0
09/03/04 18:08:38 INFO mapred.JobClient:     Combine input records=0
09/03/04 18:08:38 INFO mapred.JobClient:     Map output records=5000
09/03/04 18:08:38 INFO mapred.JobClient:     Reduce input records=5000



> [Performance]Combine multi-mapreduce jobs into a single mapreduce job while computing
such as 'C=alpha*B + A' 
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: HAMA-154
>                 URL: https://issues.apache.org/jira/browse/HAMA-154
>             Project: Hama
>          Issue Type: Improvement
>          Components: implementation
>    Affects Versions: 0.1.0
>            Reporter: Samuel Guo
>             Fix For: 0.1.0
>
>         Attachments: HAMA-154.patch, HAMA-154_v02.patch
>
>
> now, In Hama we use two map-reduce jobs to do 'C=alpha*B + A'.
> first job is to generate a matrix B' = alpha * B
> second job is to generate the result matrix C = B' + A
> I think the first map-reduce job is unneccesary for we can do the constant-multiplication
during Addition-MapReducing.
> And we also encount the same problem during computing 'A1 + A2 + ... + An'.
> we need to generate (n-1) hbase-tables for the temp results during addition. Can we just
combine n-1 addition-mapreduce jobs into only a simple one?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message