hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Panshul Whisper <ouchwhis...@gmail.com>
Subject Re: Spring for hadoop
Date Wed, 23 Jan 2013 21:55:26 GMT
Hello Radim,

Your solution sounds interesting. Is it possible for me to try the solution
before I buy it?

Thnx ,

On Wed, Jan 23, 2013 at 1:07 AM, Radim Kolar <hsn@filez.com> wrote:

> i have solution integrating spring beans and spring batch directly into
> hadoop core. its far more advanced then spring data hadoop support with
> pojo patch.
> in my solution every component of mapreduce can be hadoop bean. You will
> get spring batch integrated directly into mapper, which means that you can
> run multiple steps in one mapper pass and because of async write done by
> spring batch you will get about 3x higher write performance. i have
> rewriten HDFS which has way faster writes. Spring batch component replaces
> standard hadoop job manager (that thing with web gui) and spring
> integration is used for advanced stuff like multiresource scheduling. You
> can write simple java bean for every new resource you want to add into
> system and another bean for logic for assigning jobs based on that
> particular resource.
> I submitted few patches to hadoop but they were not interesting enough to
> get into core. If you want to buy my hadoop with integrated spring, let me
> know.

Ouch Whisper

View raw message