hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HAMA-107) Blocking job should be a map/reduce job
Date Wed, 26 Nov 2008 03:06:44 GMT

     [ https://issues.apache.org/jira/browse/HAMA-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Edward J. Yoon updated HAMA-107:
--------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I just committed this, Thanks, samuel.

And, I tested 1000 * 1000 matrices multiplication. 

----
[d8g053:/root/hama-trunk]# bin/hama examples mult 1000 1000
08/11/26 12:01:09 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/26 12:01:16 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randrfczm
08/11/26 12:01:16 INFO hama.AbstractMatrix: Create the 1000 * 1000 random matrix : DenseMatrix_randrfczm
08/11/26 12:01:27 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/26 12:01:33 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randzzdtj
08/11/26 12:01:33 INFO hama.AbstractMatrix: Create the 1000 * 1000 random matrix : DenseMatrix_randzzdtj
08/11/26 12:01:45 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
08/11/26 12:01:45 WARN mapred.JobClient: Use genericOptions for the option -libjars
08/11/26 12:01:45 WARN mapred.JobClient: No job jar file set.  User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
08/11/26 12:01:45 INFO mapred.JobClient: Running job: job_200811201535_0009
08/11/26 12:01:46 INFO mapred.JobClient:  map 0% reduce 0%
08/11/26 12:01:54 INFO mapred.JobClient:  map 50% reduce 0%
08/11/26 12:01:59 INFO mapred.JobClient:  map 100% reduce 0%
08/11/26 12:02:02 INFO mapred.JobClient:  map 100% reduce 50%
08/11/26 12:02:04 INFO mapred.JobClient:  map 100% reduce 58%
08/11/26 12:02:12 INFO mapred.JobClient: Job complete: job_200811201535_0009
08/11/26 12:02:12 INFO mapred.JobClient: Counters: 13
08/11/26 12:02:12 INFO mapred.JobClient:   File Systems
08/11/26 12:02:12 INFO mapred.JobClient:     Local bytes read=33195128
08/11/26 12:02:12 INFO mapred.JobClient:     Local bytes written=66390384
08/11/26 12:02:12 INFO mapred.JobClient:   Job Counters 
08/11/26 12:02:12 INFO mapred.JobClient:     Launched reduce tasks=2
08/11/26 12:02:12 INFO mapred.JobClient:     Launched map tasks=2
08/11/26 12:02:12 INFO mapred.JobClient:   Map-Reduce Framework
08/11/26 12:02:12 INFO mapred.JobClient:     Reduce input groups=4
08/11/26 12:02:12 INFO mapred.JobClient:     Combine output records=0
08/11/26 12:02:12 INFO mapred.JobClient:     Map input records=1000
08/11/26 12:02:12 INFO mapred.JobClient:     Reduce output records=0
08/11/26 12:02:12 INFO mapred.JobClient:     Map output bytes=32929780
08/11/26 12:02:12 INFO mapred.JobClient:     Map input bytes=0
08/11/26 12:02:12 INFO mapred.JobClient:     Combine input records=0
08/11/26 12:02:12 INFO mapred.JobClient:     Map output records=2000
08/11/26 12:02:12 INFO mapred.JobClient:     Reduce input records=2000
08/11/26 12:02:12 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
08/11/26 12:02:12 WARN mapred.JobClient: Use genericOptions for the option -libjars
08/11/26 12:02:13 WARN mapred.JobClient: No job jar file set.  User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
08/11/26 12:02:13 INFO mapred.JobClient: Running job: job_200811201535_0010
08/11/26 12:02:14 INFO mapred.JobClient:  map 0% reduce 0%
08/11/26 12:02:25 INFO mapred.JobClient:  map 50% reduce 0%
08/11/26 12:02:26 INFO mapred.JobClient:  map 100% reduce 0%
08/11/26 12:02:37 INFO mapred.JobClient: Job complete: job_200811201535_0010
08/11/26 12:02:37 INFO mapred.JobClient: Counters: 13
08/11/26 12:02:37 INFO mapred.JobClient:   File Systems
08/11/26 12:02:37 INFO mapred.JobClient:     Local bytes read=33195128
08/11/26 12:02:37 INFO mapred.JobClient:     Local bytes written=66390384
08/11/26 12:02:37 INFO mapred.JobClient:   Job Counters 
08/11/26 12:02:37 INFO mapred.JobClient:     Launched reduce tasks=2
08/11/26 12:02:37 INFO mapred.JobClient:     Launched map tasks=2
08/11/26 12:02:37 INFO mapred.JobClient:   Map-Reduce Framework
08/11/26 12:02:37 INFO mapred.JobClient:     Reduce input groups=4
08/11/26 12:02:37 INFO mapred.JobClient:     Combine output records=0
08/11/26 12:02:37 INFO mapred.JobClient:     Map input records=1000
08/11/26 12:02:37 INFO mapred.JobClient:     Reduce output records=0
08/11/26 12:02:37 INFO mapred.JobClient:     Map output bytes=32929780
08/11/26 12:02:37 INFO mapred.JobClient:     Map input bytes=0
08/11/26 12:02:37 INFO mapred.JobClient:     Combine input records=0
08/11/26 12:02:37 INFO mapred.JobClient:     Map output records=2000
08/11/26 12:02:37 INFO mapred.JobClient:     Reduce input records=2000
08/11/26 12:02:37 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/26 12:02:43 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randjbovq
08/11/26 12:02:43 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
08/11/26 12:02:43 WARN mapred.JobClient: Use genericOptions for the option -libjars
08/11/26 12:02:43 WARN mapred.JobClient: No job jar file set.  User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
08/11/26 12:02:43 INFO mapred.JobClient: Running job: job_200811201535_0011
08/11/26 12:02:44 INFO mapred.JobClient:  map 0% reduce 0%
08/11/26 12:02:50 INFO mapred.JobClient:  map 50% reduce 0%
08/11/26 12:02:52 INFO mapred.JobClient:  map 100% reduce 0%
08/11/26 12:03:02 INFO mapred.JobClient:  map 100% reduce 50%
08/11/26 12:03:04 INFO mapred.JobClient:  map 100% reduce 87%
08/11/26 12:03:46 INFO mapred.JobClient: Job complete: job_200811201535_0011
08/11/26 12:03:46 INFO mapred.JobClient: Counters: 13
08/11/26 12:03:46 INFO mapred.JobClient:   File Systems
08/11/26 12:03:46 INFO mapred.JobClient:     Local bytes read=16166452
08/11/26 12:03:46 INFO mapred.JobClient:     Local bytes written=32333028
08/11/26 12:03:46 INFO mapred.JobClient:   Job Counters 
08/11/26 12:03:46 INFO mapred.JobClient:     Launched reduce tasks=2
08/11/26 12:03:46 INFO mapred.JobClient:     Launched map tasks=2
08/11/26 12:03:46 INFO mapred.JobClient:   Map-Reduce Framework
08/11/26 12:03:46 INFO mapred.JobClient:     Reduce input groups=2
08/11/26 12:03:46 INFO mapred.JobClient:     Combine output records=0
08/11/26 12:03:46 INFO mapred.JobClient:     Map input records=2
08/11/26 12:03:46 INFO mapred.JobClient:     Reduce output records=2000
08/11/26 12:03:46 INFO mapred.JobClient:     Map output bytes=16041064
08/11/26 12:03:46 INFO mapred.JobClient:     Map input bytes=0
08/11/26 12:03:46 INFO mapred.JobClient:     Combine input records=0
08/11/26 12:03:46 INFO mapred.JobClient:     Map output records=8
08/11/26 12:03:46 INFO mapred.JobClient:     Reduce input records=8
08/11/26 12:03:57 INFO client.HBaseAdmin: Disabled DenseMatrix_randrfczm
08/11/26 12:03:57 INFO client.HBaseAdmin: Deleted DenseMatrix_randrfczm
08/11/26 12:04:07 INFO client.HBaseAdmin: Disabled DenseMatrix_randzzdtj
08/11/26 12:04:07 INFO client.HBaseAdmin: Deleted DenseMatrix_randzzdtj
08/11/26 12:04:18 INFO client.HBaseAdmin: Disabled DenseMatrix_randjbovq
08/11/26 12:04:18 INFO client.HBaseAdmin: Deleted DenseMatrix_randjbovq



> Blocking job should be a map/reduce job
> ---------------------------------------
>
>                 Key: HAMA-107
>                 URL: https://issues.apache.org/jira/browse/HAMA-107
>             Project: Hama
>          Issue Type: Improvement
>          Components: implementation
>            Reporter: Edward J. Yoon
>            Assignee: Samuel Guo
>             Fix For: 0.1.0
>
>         Attachments: HAMA-107.patch, HAMA-107_v02.patch, HAMA-107_v03.patch
>
>
> DenseMatrix.blocking() used for build a blocked dense matrix. But, it should be work
on map/reduce to improve speed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message