Return-Path: Delivered-To: apmail-incubator-hama-user-archive@minotaur.apache.org Received: (qmail 20838 invoked from network); 4 Jun 2009 02:25:18 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 4 Jun 2009 02:25:18 -0000 Received: (qmail 37928 invoked by uid 500); 4 Jun 2009 02:25:30 -0000 Delivered-To: apmail-incubator-hama-user-archive@incubator.apache.org Received: (qmail 37907 invoked by uid 500); 4 Jun 2009 02:25:30 -0000 Mailing-List: contact hama-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hama-user@incubator.apache.org Delivered-To: mailing list hama-user@incubator.apache.org Received: (qmail 37897 invoked by uid 99); 4 Jun 2009 02:25:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Jun 2009 02:25:30 +0000 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.216.201] (HELO mail-px0-f201.google.com) (209.85.216.201) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Jun 2009 02:25:19 +0000 Received: by pxi39 with SMTP id 39so398584pxi.32 for ; Wed, 03 Jun 2009 19:24:58 -0700 (PDT) MIME-Version: 1.0 Sender: edward@udanax.org Received: by 10.142.180.10 with SMTP id c10mr59185wff.316.1244082297967; Wed, 03 Jun 2009 19:24:57 -0700 (PDT) In-Reply-To: References: Date: Thu, 4 Jun 2009 11:24:57 +0900 X-Google-Sender-Auth: 41d1d39545cb7f32 Message-ID: Subject: Re: Hama status / From: "Edward J. Yoon" To: hama-user@incubator.apache.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Yes, the goal is to handle really huge matrices, for example, matrix operations for large-scale statistical processing, matrix decomposition of huge web link graph/social graph. It's the tests on 5 nodes and 10 nodes. In the future, I'll try them on a thousand nodes. On Thu, Jun 4, 2009 at 1:19 AM, tog wrote: > Hi Edward, > > I had a look to the benchmarks ... > Well a 5000 dense matrix multiply is roughly 30 seconds on my laptop. I h= ave > been doing out-of-core parallel matrix factor on solve with dense systems= up > to 350000 > so I guess this is at least probably for larger matrix that Hama could be > interesting > Do you plan to do such tests with really huge matrices ? > Otherwise what is your business case ? > > Cheers > Guillaume > > On Wed, Jun 3, 2009 at 6:14 PM, Edward J. Yoon wro= te: > >> FYI, I ran some benchmarks - >> http://wiki.apache.org/hama/PerformanceEvaluation >> >> If you need any help, Pls let us know. >> >> Thanks. >> >> On Wed, Jun 3, 2009 at 6:55 PM, tog wrote: >> > Yes I understand the difference between MPI and Hadoop - I have been >> using >> > MPI before it actually exists :) >> > But as you phrased it, I had the impression that Hama was working on a= 1 >> > node/core cluster !! >> > >> > Regards >> > Guillaume >> > >> > On Wed, Jun 3, 2009 at 5:44 PM, Edward J. Yoon > >wrote: >> > >> >> Hi, >> >> >> >> There is some difference between Map/Reduce and MPI programming. MPI >> >> is based on and designed for fast parallel computing using network >> >> communication on small cluster. Since MPI requires network >> >> communication, Increased node numbers, there is a linear increase of >> >> network cost at same time. On the contrary, Map/Reduce is designed to >> >> distributed processing by connecting many commodity computers >> >> together. Therefore, The algorithms should avoid large amounts of >> >> communication for best performance and that key is the 'sequential >> >> process'. >> >> >> >> Thanks. >> >> >> >> On Wed, Jun 3, 2009 at 6:07 PM, tog wrot= e: >> >> > Hi Edward >> >> > >> >> > I have a test to do which is basically Sparce Mat Vec multiplicatio= n >> and >> >> Mat >> >> > norm computation. So that should be possible with Hama in its curre= nt >> >> state >> >> > I guess. >> >> > What do you mean by "sequentially executed" >> >> > >> >> > Cheers >> >> > Guillaume >> >> > >> >> > On Wed, Jun 3, 2009 at 5:00 PM, Edward J. Yoon > >> >wrote: >> >> > >> >> >> Hi, >> >> >> >> >> >> Currently, the basic matrix operations are implemented based on th= e >> >> >> map/reduce programming model. For example, the matrix get/set >> methods, >> >> >> the matrix norms, matrix-matrix multiplication/addition, matrix >> >> >> transpose. In near future, SVD, Eigenvalue decomposition and some >> >> >> graph algorithms will be implemented. All the operations are >> >> >> sequentially executed. >> >> >> >> >> >> Thanks. >> >> >> >> >> >> On Wed, Jun 3, 2009 at 5:45 PM, tog >> wrote: >> >> >> > >> >> >> > Hi, >> >> >> > >> >> >> > I would like to know what is the status of Hama ? >> >> >> > What am I able to do with it ? >> >> >> > What are the future directions ? >> >> >> > >> >> >> > Cheers >> >> >> > Guillaume >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Best Regards, Edward J. Yoon @ NHN, corp. >> >> >> edwardyoon@apache.org >> >> >> http://blog.udanax.org >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > >> >> > PGP KeyID: 1024D/47172155 >> >> > FingerPrint: C739 8B3C 5ABF 127F CCFA =C2=A05835 F673 370B 4717 215= 5 >> >> > >> >> > http://cheztog.blogspot.com >> >> > >> >> >> >> >> >> >> >> -- >> >> Best Regards, Edward J. Yoon @ NHN, corp. >> >> edwardyoon@apache.org >> >> http://blog.udanax.org >> >> >> > >> > >> > >> > -- >> > >> > PGP KeyID: 1024D/47172155 >> > FingerPrint: C739 8B3C 5ABF 127F CCFA =C2=A05835 F673 370B 4717 2155 >> > >> > http://cheztog.blogspot.com >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon @ NHN, corp. >> edwardyoon@apache.org >> http://blog.udanax.org >> > > > > -- > > PGP KeyID: 1024D/47172155 > FingerPrint: C739 8B3C 5ABF 127F CCFA =C2=A05835 F673 370B 4717 2155 > > http://cheztog.blogspot.com > --=20 Best Regards, Edward J. Yoon @ NHN, corp. edwardyoon@apache.org http://blog.udanax.org