hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Hadoop and Matlab
Date Wed, 22 Apr 2009 01:40:25 GMT
Where to store the images? How to retrieval the images?

If you have a metadata for the images, the map task can receives a
'filename' of image as a key, and file properies (host, file path,
..,etc) as its value. Then, I guess you can handle the matlab process
using runtime object on hadoop cluster.

On Wed, Apr 22, 2009 at 9:30 AM, Sameer Tilak <sameer.ucsd@gmail.com> wrote:
> Hi Edward,
> Yes, we're building this for handling hundreds of thousands images (at
> least). We're thinking processing of individual images (or a set of images
> together) will be done in Matlab itself. However, we can use Hadoop
> framework to process the data in parallel fashion. One Matlab instance
> handling few hundred images (as a mapper) and have hundreds of such
> instances and then combine (reducer) the o/p of each instance.
> On Tue, Apr 21, 2009 at 5:06 PM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>> Hi, What is the input data?
>> According to my understanding, you have a lot of images and want to
>> process all images using your matlab script. Then, You should write
>> some code yourself. I did similar thing for plotting graph with
>> gnuplot. However, If you want to do large-scale linear algebra
>> operations for large image processing, I would recommend investigating
>> other solutions. Hadoop is not a general purpose clustering software,
>> and it cannot run matlab.
>> On Wed, Apr 22, 2009 at 2:55 AM, Sameer Tilak <sameer.ucsd@gmail.com>
>> wrote:
>> > Hi there,
>> >
>> > We're working on an image analysis project. The image processing code is
>> > written in Matlab. If I invoke that code from a shell script and then use
>> > that shell script within Hadoop streaming, will that work? Has anyone
>> done
>> > something along these lines?
>> >
>> > Many thaks,
>> > --ST.
>> >
>> --
>> Best Regards, Edward J. Yoon
>> edwardyoon@apache.org
>> http://blog.udanax.org

Best Regards, Edward J. Yoon

View raw message