Mailing-List: contact commits-help@hama.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hama.apache.org
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: Apache Wiki <wikidiffs@apache.org>
To: Apache Wiki <wikidiffs@apache.org>
Date: Sat, 18 Aug 2012 15:52:02 -0000
Message-ID: <20120818155202.47353.14747@eos.apache.org>
Subject: 
 =?utf-8?q?=5BHama_Wiki=5D_Update_of_=22SpMV=22_by_Mikalai_Parafeniuk?=
Auto-Submitted: auto-generated

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for chan=
ge notification.

The "SpMV" page has been changed by Mikalai Parafeniuk:
http://wiki.apache.org/hama/SpMV?action=3Ddiff&rev1=3D11&rev2=3D12

  /user/hduser/spmv/vector-seq
  /user/hduser/spmv/vector-txt
  }}}
- Suffix `seq` denotes that directory contains sequence files. Suffix `txt`=
 denotes that directory contains human-readable text files. =

+ Suffix `seq` denotes that directory contains sequence files. Suffix `txt`=
 denotes that directory contains human-readable text files. <<BR>>
+ Also I defined some shell variables in .bashrc file of hadoop user to sim=
plify following scripts.
+ {{{
+ export HAMA_EXAMPLES=3D$HAMA_HOME/hama-examples*.jar
+ export SPMV=3D/user/hduser/spmv
+ }}}
+ First variable allows fast access to jar with hama examples, which plased=
 in hama home directory, second variable is prefix in HDFS for tests in thi=
s tutorial. If you not defined this variables just substitute appropriate v=
alues into following scripts.
  =3D=3D=3D=3D Usage with RandomMatrixGenerator =3D=3D=3D=3D
  `RandomMatrixGenerator` as a `SpMV` works with sequence file format. So, =
to multiply random matrix with random vector we will do the following: gene=
rate matrix and vector; convert matrix, vector and result to text file; vie=
w matrix, vector and result. This sequence is described by the following co=
de snippet:
  {{{
- 1: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar rmgenerator /user/hduser=
/spmv/matrix-seq 4 4 0.3 4
- 2: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar rmgenerator /user/hduser=
/spmv/vector-seq 1 4 0.8 1
- 3: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar spmv /user/hduser/spmv/m=
atrix-seq /user/hduser/spmv/vector-seq /user/hduser/spmv/result-seq 4
- 4: ../hadoop/bin/hadoop dfs -rmr /user/hduser/spmv/result-seq/part
- 5: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar matrixtotext /user/hduse=
r/spmv/matrix-seq /user/hduser/spmv/matrix-txt
- 6: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar matrixtotext /user/hduse=
r/spmv/vector-seq /user/hduser/spmv/vector-txt
- 7: bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar matrixtotext /user/hduse=
r/spmv/result-seq /user/hduser/spmv/result-txt
+ 0:    hadoop dfs -rmr $SPMV/*/*
+ 1:    hama jar $HAMA_EXAMPLES rmgenerator $SPMV/matrix-seq 6 6 0.4 4
+ 2:    hama jar $HAMA_EXAMPLES rmgenerator $SPMV/vector-seq 1 6 0.9 4
+ 3:    hama jar $HAMA_EXAMPLES spmv $SPMV/matrix-seq $SPMV/vector-seq $SPM=
V/result-seq 4
+ 4:    hadoop dfs -rmr $SPMV/result-seq/part
+ 5:    hama jar $HAMA_EXAMPLES matrixtotext $SPMV/matrix-seq $SPMV/matrix-=
txt
+ 6:    hama jar $HAMA_EXAMPLES matrixtotext $SPMV/vector-seq $SPMV/vector-=
txt
+ 7:    hama jar $HAMA_EXAMPLES matrixtotext $SPMV/result-seq $SPMV/result-=
txt
- 8: ../hadoop/bin/hadoop dfs -cat /user/hduser/spmv/matrix-txt/*
+ 8:    hadoop dfs -cat /user/hduser/spmv/matrix-txt/*
-     0	 4 1 2 0.09775514911904559
-     3	 4 1 1 0.22718006778335464
-     1	 4 1 1 0.796916052801057
-     3	 4 1 2 0.9680719390036476
-     2	 4 1 0 0.44269679525022
+    0	 6 3 5 0.24316243288531214 2 0.638622414091597 3 0.5480468710898891
+    3	 6 2 5 0.5054043538570098 2 0.03911646523753309
+    1	 6 3 4 0.5077528966368161 5 0.5780340816354201 3 0.4626752204959449
+    4	 6 2 1 0.6512355661856207 4 0.08804976645891671
+    2	 6 2 4 0.7200271909735554 1 0.3510851368183805
+    5	 6 2 2 0.5848717104309032 3 0.0889791409798859
+ =

- 9: ../hadoop/bin/hadoop dfs -cat /user/hduser/spmv/vector-txt/*
+ 9:    hadoop dfs -cat /user/hduser/spmv/vector-txt/*
-     0	 4 4 0 0.3111172930766064 1 0.27242674637140374 2 0.113947467640978=
63 3 0.0
+    0	 6 6 0 0.3365077672167889 1 0.17498609722570935 2 0.3280641095064884=
5 3 0.6016567879100464 4 0.786158850847722 5 0.6856872945972037
- 10: ../hadoop/bin/hadoop dfs -cat /user/hduser/spmv/result-txt/*
+ 10:   hadoop dfs -cat /user/hduser/spmv/result-txt/*
-     0	 5 5 0 0.011138951690981488 1 0.21710124739573375 2 0.1377306285919=
371 3 0.11030934594375758 4 0.0
+    0	 6 6 0 0.7059786044267415 1 1.0738967463653346 2 0.6274907669206862 =
3 0.35938205240905363 4 0.18317827331814918 5 0.24541032101100438
  }}}
- Line 1-2: Generation of input matrix and vector.<<BR>>
+ Line 0: Clean up of directories related to SpMV tests.<<BR>>
+ Line 1-2: Generation of input matrix and vector. In this example we test =
6x6 matrix and 1x6 vector multiplication<<BR>>
  Line 3: SpMV algorithm.<<BR>>
  Line 4: Deletion of part files from output directory at line 4. NOTE: `ma=
trixtotext` will fail if this step will not be performed, because `result-s=
eq` will containg part folder and `matrixtotext` don't know how to deal wit=
h it yet.<<BR>>
  Line 5-7: Convertion of input matrix, input vector and result to text for=
mat.<<BR>>
- Line 8-10: Showing the result. NOTE: currently SpMV is buggy - you can se=
e, that output vector has length of 5, and the last cell of vector has inco=
rrect value. This will be fixed soon.
+ Line 8-10: Showing the result. =

  =

  =3D=3D=3D=3D Usage with arbitrary text files =3D=3D=3D=3D
  SpMV works with `SequenceFile`, so we need to provide tools to convert in=
put and output of SpMV between sequence file format and text format. These =
tools are `matrixtoseq` and `matrixtotext`. This programs are included in e=
xample driver, so they can be launched like any other example. `matrixtoseq=
` converts matrix, represented in text file to sequence file format. Also t=
his program gives choice to choose target writable: `DenseVectorWritable` a=
nd `SparseVectorWritable`.