hdt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeffrey Zemerick <jzemer...@apache.org>
Subject Re: Plugin organization and build system
Date Fri, 11 Jan 2013 15:17:17 GMT
Just to expand on my Tycho +1, I had good success with it for a
project containing about 20 plugins, including their associated
features, product, and update site projects. Building it all at once
is very easy now and Maven can deploy the newly built update site when
needed. (I'm not an Ant person but I figure Ant can probably do that
as well.) The only difficulties has been with the pom-first versus
manifest-first projects. If I remember right I don't recall many
dependencies of the HDT so maybe it won't be a problem (but if so
perhaps someone out there will know a better way to manage it).

Jeff


On Fri, Jan 11, 2013 at 12:07 AM, Bob Kerns <rwk@acm.org> wrote:
> I'll have a look at Tycho, I always like new tools. But I haven't found
> pure ant plugin construction to be very difficult, so long as you steer
> 100% clear of anything provided by the PDE!
>
> One goal of whatever build technology we adopt should be to make
> reproducible setup and build as simple and clear as possible. Build setup
> can be a barrier to project involvement.
>
> Personally, I always find maven itself to be such a barrier. I'm willing to
> take it as a given, though, given its widespread adoption and use with
> Hadoop. I presume this also helps with integration with the Apache build
> and distribution environment?
>
> I'll try to keep us honest with the
> "without having to do a bunch of local setup" principle; somehow I always
> end up getting involved in build systems, because they're important.
>
> Your plugin naming structure looks good, but if we're to support multiple
> versions of Hadoop, I think we need a connector architecture, Maybe
> org.apache.hdt.connector.<target> would
> be a good place to put them?
>
> Thanks for getting the source up there. I notice there's no history? Do
> people think that's important? Would it help us with looking at what we
> need to do to support older versions of Hadoop? (I wasn't successful in
> pulling in the history in my attempts, and I'm not eager to stall the
> project trying to pull it in, since it's available in SVN. Perhaps we can
> provide a document that describes where to find the various historical
> versions?)
>
>
> On Thu, Jan 10, 2013 at 11:28 AM, Jeffrey Zemerick <jzemerick@apache.org>wrote:
>
>> Sounds fine.
>>
>> +1 for using Tycho. It's much simpler.
>>
>> Jeff
>>
>>
>> On Thu, Jan 10, 2013 at 2:11 PM, Adam Berry <adamb@apache.org> wrote:
>> >
>> > Hi everyone,
>> >
>> > First, I've dropped the code from Hadoop contrib into our git repo, its
>> on
>> > its own branch, hadoop-contrib. The reason I put it on a branch is
>> because
>> > I think that splitting things up a little would be a good idea, and
>> should
>> > make it a little easier to support multiple versions of Hadoop.
>> >
>> > So, the tools as they stand are just in one plugin. Broadly, the features
>> > right now can be divided into;
>> >
>> > MapReduce project and class code support (wizards etc)
>> > Launch support for Hadoop
>> > HDFS interaction
>> >
>> > So taking a root name space of org.apache.hdt, I suggest something like
>> the
>> > following for the plugin names
>> >
>> > org.apache.hdt.core
>> >
>> > org.apache.hdt.ui
>> >
>> > org.apache.hdt.debug.core
>> > org.apache.hdt.debug.ui
>> >
>> > org.apache.hdt.hdfs.core
>> > org.apache.hdt.hdfs.ui
>> >
>> > org.apache.hdt.help
>> >
>> > These may be a little fluid as we get into the details here, but from
>> 10000
>> > feet it looks ok.
>> >
>> > Finally, I would also like to suggest Tycho (Maven plugin for doing
>> Eclipse
>> > build stuff) as our build tool. I've done my fair share of pure Ant PDE
>> > build stuff over the years, and Tycho is vastly easier, and would make it
>> > much easier for people to build themselves without having to do a bunch
>> of
>> > local setup first.
>> >
>> > Thoughts? If everyone thinks these are ok, I'll enter some issues and get
>> > cracking.
>> >
>> > Cheers,
>> > Adam
>>

Mime
View raw message