hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Costin Leau <costin.l...@gmail.com>
Subject Re: Hadoop with Spring / Guice
Date Fri, 30 Dec 2011 11:19:59 GMT
Hi,

This tends to be a problem with any library that you use inside Hadoop.

One of the easiest way to distribute your dependencies (w/o managing) is
to put them under lib/ folder in your job and Hadoop will take care of
the rest - if there are any better suggestions, I'd be interested in
hearing about them.

Cheers,

On 12/30/2011 12:56 PM, Eyal Golan wrote:
> Thank you Costin.
> 
> I have worked with Spring before but never set up a project.
> So I guess I'll need to do a little bit more ramp-up to use Spring-Hadoop.
> 
> Another issue that concerns me is the Hadoop cluster management.
> I don't have access for the cluster, besides adding my job and use some
> kind of CLI to schedule / trigger the jobs.
> 
> I am wondering whether I can create a Hadoop-Spring application without
> managing the cluster.
> 
> 
> Thanks,
> 
> Eyal
> 
> Eyal Golan
> egolan74@gmail.com <mailto:egolan74@gmail.com>
> 
> Visit: http://jvdrums.sourceforge.net/
> LinkedIn: http://www.linkedin.com/in/egolan74
> Skype: egolan74
> 
> P  Save a tree. Please don't print this e-mail unless it's really necessary
> 
> 
> 
> On Fri, Dec 30, 2011 at 11:53 AM, Costin Leau <costin.leau@gmail.com
> <mailto:costin.leau@gmail.com>> wrote:
> 
>     Hi,
> 
>     My name is Costin Leau and I'm the lead of Spring Hadoop (SHDP) project.
> 
>     SHDP provides DI support allowing basic POJOs to be used as
>     mapper/reducers. This feature is currently developed on a dedicated
>     branch [1] and we plan to merge it in master in the near future.
> 
>     In addition to the pojo/DI support, you might be interested in the SHDP
>     integration with HBase/Hive/Pig - see [2]
> 
>     Everything is open-source, under ASL 2, available on github. We plan to
>     release the milestone in Jan/Feb 2012.
> 
>     Feedback and ideas are more then welcome!
> 
>     [1] https://github.com/SpringSource/spring-hadoop/tree/mr-pojo
>     [2] http://static.springsource.org/spring-hadoop/docs/current/reference/
> 
>     Hope this helps,
>     --
>     Costin
> 
> 


-- 
Costin

Mime
View raw message