hdt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Berry <ad...@apache.org>
Subject Re: Plugin organization and build system
Date Fri, 11 Jan 2013 15:27:33 GMT
Thanks for the feedback Bob! Comments inline.

Cheers,
Adam

On Thu, Jan 10, 2013 at 11:07 PM, Bob Kerns <rwk@acm.org> wrote:

> I'll have a look at Tycho, I always like new tools. But I haven't found
> pure ant plugin construction to be very difficult, so long as you steer
> 100% clear of anything provided by the PDE!
>
> One goal of whatever build technology we adopt should be to make
> reproducible setup and build as simple and clear as possible. Build setup
> can be a barrier to project involvement.
>
> Personally, I always find maven itself to be such a barrier. I'm willing to
> take it as a given, though, given its widespread adoption and use with
> Hadoop. I presume this also helps with integration with the Apache build
> and distribution environment?
>
> I'll try to keep us honest with the
> "without having to do a bunch of local setup" principle; somehow I always
> end up getting involved in build systems, because they're important.
>

So really the reason to go with tycho is that someone coming to the
project, who would like to check things out and build would only need to do
something like;

git clone ....
mvn package

And thats it. Tycho/Maven resolves the Eclipse dependencies and everything
else, no local build setup necessary. There is a nice Tycho walkthrough
that starts at
http://codeandme.blogspot.com/2012/12/tycho-build-1-building-plug-ins.html.

Given the Apache way that the source is the release, and binaries are a
convenience, the ease here is a nice win.

Maven does take quite a bit of buying in, but the rewards are there IMHO.

>
> Your plugin naming structure looks good, but if we're to support multiple
> versions of Hadoop, I think we need a connector architecture, Maybe
> org.apache.hdt.connector.<target> would
> be a good place to put them?
>

Nice idea for the connector namespace. It wasn't in my list as I was
suggesting what to rewrite the existing code as, but it feels right, and
will also cover if we need to separate out Hadoop, MapReduce API and HDFS
multi version support (which I think we might).

>
> Thanks for getting the source up there. I notice there's no history? Do
> people think that's important? Would it help us with looking at what we
> need to do to support older versions of Hadoop? (I wasn't successful in
> pulling in the history in my attempts, and I'm not eager to stall the
> project trying to pull it in, since it's available in SVN. Perhaps we can
> provide a document that describes where to find the various historical
> versions?)
>

I also struggled with the history for a little bit, but decided to just get
it up and move on. I like the document idea, I'll add one on the branch
that details this out.

>
>
> On Thu, Jan 10, 2013 at 11:28 AM, Jeffrey Zemerick <jzemerick@apache.org
> >wrote:
>
> > Sounds fine.
> >
> > +1 for using Tycho. It's much simpler.
> >
> > Jeff
> >
> >
> > On Thu, Jan 10, 2013 at 2:11 PM, Adam Berry <adamb@apache.org> wrote:
> > >
> > > Hi everyone,
> > >
> > > First, I've dropped the code from Hadoop contrib into our git repo, its
> > on
> > > its own branch, hadoop-contrib. The reason I put it on a branch is
> > because
> > > I think that splitting things up a little would be a good idea, and
> > should
> > > make it a little easier to support multiple versions of Hadoop.
> > >
> > > So, the tools as they stand are just in one plugin. Broadly, the
> features
> > > right now can be divided into;
> > >
> > > MapReduce project and class code support (wizards etc)
> > > Launch support for Hadoop
> > > HDFS interaction
> > >
> > > So taking a root name space of org.apache.hdt, I suggest something like
> > the
> > > following for the plugin names
> > >
> > > org.apache.hdt.core
> > >
> > > org.apache.hdt.ui
> > >
> > > org.apache.hdt.debug.core
> > > org.apache.hdt.debug.ui
> > >
> > > org.apache.hdt.hdfs.core
> > > org.apache.hdt.hdfs.ui
> > >
> > > org.apache.hdt.help
> > >
> > > These may be a little fluid as we get into the details here, but from
> > 10000
> > > feet it looks ok.
> > >
> > > Finally, I would also like to suggest Tycho (Maven plugin for doing
> > Eclipse
> > > build stuff) as our build tool. I've done my fair share of pure Ant PDE
> > > build stuff over the years, and Tycho is vastly easier, and would make
> it
> > > much easier for people to build themselves without having to do a bunch
> > of
> > > local setup first.
> > >
> > > Thoughts? If everyone thinks these are ok, I'll enter some issues and
> get
> > > cracking.
> > >
> > > Cheers,
> > > Adam
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message