hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Image indexing/searching with Hadoop and MPI
Date Wed, 03 Jun 2009 09:17:49 GMT
> This is a kind of newbie question (at least as far as Hadoop is concerned).
> I was wondering if they were any Hadoop based project around dealing with
> Image indexing and searching ? We are working is this area and might be
> interesting to have a look in such a project.

There is a text-search engine library, called lucene. See also the
nutch project. Otherwise, Did you mean something like content-based
image indexing and searching usig image attributes, such as, color,
texture, and etc., not the text of image tag?

> Second question is dealing with scientific computing with Haddop. Does
> anyone has try to use Hadoop to parallelize a scientific application ? I
> know there is Hama but it does not seem very active these days (I might be
> wrong ;) )
> Some time ago, I heard of an attempt of implementing some MPI implementation
> on top of Hadoop , was it really the plan, is there any update ?
> Anyway, I would be interested in any paper/fedeback on the performance of
> scientific application running on large clusters using Hadoop.

I think the MPI programming isn't suitable for the concept of
distributed hdfs and map/reduce programming system, since MPI requires
the heavy communication among the nodes.

FYI, In hama, currently the basic matrix operations are implemented
based on the map/reduce programming model. For example, the matrix
get/set methods, the matrix norms, matrix-matrix
multiplication/addition, matrix transpose. In near future, SVD,
Eigenvalue decomposition and some graph algorithms will be
implemented. All the operations are sequentially executed.

Thanks.

On Wed, Jun 3, 2009 at 5:32 PM, tog <guillaume.alleon@gmail.com> wrote:
> Hi there,
>
> This is a kind of newbie question (at least as far as Hadoop is concerned).
> I was wondering if they were any Hadoop based project around dealing with
> Image indexing and searching ? We are working is this area and might be
> interesting to have a look in such a project.
> Second question is dealing with scientific computing with Haddop. Does
> anyone has try to use Hadoop to parallelize a scientific application ? I
> know there is Hama but it does not seem very active these days (I might be
> wrong ;) )
> Some time ago, I heard of an attempt of implementing some MPI implementation
> on top of Hadoop , was it really the plan, is there any update ?
> Anyway, I would be interested in any paper/fedeback on the performance of
> scientific application running on large clusters using Hadoop.
>
> Best Regards
> Guillaume
>



-- 
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org

Mime
View raw message