hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Hadoop testing project
Date Fri, 18 Feb 2011 16:24:19 GMT
On 17/02/2011 18:36, Eric Yang wrote:

> I think the bigger concern is that Hadoop ecosystem does not have a standard
> method in linking dependencies.  Hbase depends on Zookeeper, and Pig depends
> on Hadoop and Hbase.  Then pig decided to put hadoop-core jar in it's own
> jar file.  Chukwa depends on pig + hbase + hadoop and zookeeper.  The
> version incompatibility is probably what driving people nuts.  Hence, there
> is a new proposal on how to integrate among hadoop ecosystem.  I urge
> project owners to review the proposal and provide feedbacks.
> The proposal is located at:
> https://issues.apache.org/jira/secure/attachment/12470823/deployment.pdf
> The related jiras are:
> https://issues.apache.org/jira/browse/HADOOP-6255
> https://issues.apache.org/jira/browse/PIG-1857
> There are plans to file more jiras for related projects.

I think a focus on RPMs/debs and then say "local VM for testing", 
because that gets people used to deploying on Linux from day one, rather 
than developing on windows and being surprised that things work 
differently in production. You still have the problem of debugging 
remotely to a VM in the VLAN, but that's tractable.

> The integration
> would also be a lot easier if all related projects are using maven for
> dependency management.

Do you mean Maven the tool or the maven repo format?

View raw message