hdt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srimanth Gunturi <srima...@gmail.com>
Subject Re: Maven build
Date Mon, 30 Sep 2013 17:09:55 GMT
Hi Rahul,
HDFS needs file-system like interactions, and hence its most useful in the
project explorer as a regular project.

ZK on the other hand is more a hierarchy of nodes/models, hence can be
shown in the servers view.
Regards,
Srimanth





On Mon, Sep 30, 2013 at 12:29 AM, Rahul Sharma <rahul0208@gmail.com> wrote:

> Hi Srimanth,
>
> I am able to connect the servers(HDFS/Zk) and explore them. It works as
> expected.J
>
> There is one thing that I noticed, the HDFS server is getting created in
> the Project Explorer while the ZK one gets created in Hadoop Servers. Is
> this expected ?  shouldn't both of them be in Hadoop Servers panel ? I
> thought of checking with you before opening a jira.
>
> regards
> Rahul
>
>
> On Thu, Sep 26, 2013 at 2:06 PM, Srimanth Gunturi <srimanth@gmail.com
> >wrote:
>
> > Hello,
> > Requesting review and commit of patch attached to
> > https://issues.apache.org/jira/browse/HDT-40.
> > Was able to connect to ZK and HDFS services, and browse as expected.
> > Would be great if others could verify it works for them too.
> > Regards,
> > Srimanth
> >
> >
> >
> >
> >
> > On Tue, Sep 17, 2013 at 12:39 AM, Srimanth Gunturi <srimanth@gmail.com
> > >wrote:
> >
> > > Hello,
> > > With Rahul's excellent work in getting maven build issues resolved, we
> > > will be able to get back to feature development.
> > > A good release point for the project would be to have all the features
> of
> > > HDT and Hadoop-Eclipse projects working.
> > >
> > > The big missing piece of this merge is the existing MapReduce
> > > functionality of HDT.
> > > I have created https://issues.apache.org/jira/browse/HDT-41 to address
> > > this issue.
> > > I would like to request assignment of HDT-41 and HDT-40 to me, as I
> have
> > > started work on it.
> > >
> > > I will upload patches as soon as I have them locally working.
> > >
> > > On a side note, I would like updating of JIRA's components to include
> > > 'common', 'zookeeper', 'core' and 'ui'.
> > > Regards,
> > > Srimanth
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Sep 10, 2013 at 1:58 AM, Rahul Sharma <rahul0208@gmail.com>
> > wrote:
> > >
> > >> Hi Adam,
> > >>
> > >> I have made a fix for the issue at hand using tycho, bundle-classpath
> > and
> > >> copy-dependencies. Basically during a maven build I am copying the
> > >> required
> > >> dependencies into a folder. These dependencies are  referenced in
> > >> MANIFEST.MF using Bundle-classpath. The copying takes place before
> tycho
> > >> kicks in and thus these are available for  the compiler. This relieves
> > us
> > >> from creation of multiple modules which need to be triggered in
> multiple
> > >> profiles. Also we  do not need to create any separate bundles as the
> > >> required libs are bundles with-in the plugin. In turn we are quite
> > >> portable, without sacrificing any configuration like
> MANIFEST,classpath
> > or
> > >> any of the ecllipse settings.
> > >>
> > >> I have uploaded  the updated patch in HDT-39[1], have a look at it and
> > let
> > >> me know your thoughts for the same.
> > >>
> > >> regards
> > >> Rahul
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://issues.apache.org/jira/secure/attachment/12602309/0001-HDT-39-Using-maven-to-build-package.patch
> > >>
> > >>
> > >>
> > >> On Tue, Sep 10, 2013 at 8:41 AM, Adam Berry <adamb@apache.org> wrote:
> > >>
> > >> > Hi Rahul,
> > >> >
> > >> > So I've been thinking about this. I know that we cannot build the
> > >> > dependencies wrapper and the IDE plugins in the same reactor build,
> > >> which
> > >> > rules out the multi module approach. I looked at the current patch
> is
> > >> up,
> > >> > which makes use of profiles to separate this, as profiles are about
> > >> build
> > >> > portability, I'm not sure thats quite right.
> > >> >
> > >> > Multi module really represents how this should be done, but as we
> > can't
> > >> use
> > >> > it I propose that we actually make a new maven project to build the
> > >> > dependency wrapper plugins, and then document that to build the
> > plugins
> > >> > requires two maven commands (at this time). Building HDT can
> reference
> > >> a p2
> > >> > repository output from the dependency build, and this can take a
> local
> > >> > path.
> > >> >
> > >> > I will work up a path with this approach, and add a comment on
> HDT-39
> > to
> > >> > this effect, and we'll see if everyone is ok with this.
> > >> >
> > >> > Cheers,
> > >> > Adam
> > >> >
> > >> >
> > >> > On Tue, Sep 3, 2013 at 10:08 PM, Rahul Sharma <rahul0208@gmail.com>
> > >> wrote:
> > >> >
> > >> > > Hi Adam,
> > >> > >
> > >> > > This is in relation to issue HDT-39[1] of hadoop-eclipse
> -merging. I
> > >> > > configured things using tycho but Srimanth had a different idea
of
> > >> > things.
> > >> > > This is due to the different approach he took for hadoop-eclipse.
> I
> > >> still
> > >> > > think the same can be achieved done using tycho. There is nothing
> > >> > stopping
> > >> > > us to create specific bundles  for different versions of hadoop,
> > >> which is
> > >> > > being done at other Apache projects like servicemix, provisonr
> etc.
> > >> > >
> > >> > > Anyways, the idea was to use maven as dumb possible, just for
> > >> > dependencies
> > >> > > management, and not to use any of tricks of the trade. I tried
to
> > dig
> > >> on
> > >> > > the possible alternatives but I do not think besides tycho there
> is
> > >> any
> > >> > > other cleaner approach. In all the rest of the approaches we
have
> to
> > >> keep
> > >> > > eclipse SDK. Apparently maven and eclipse are two separate worlds
> > >> that do
> > >> > > not converge easily.
> > >> > >
> > >> > > Let me know thoughts for the same as I think we would like to
use
> > >> maven
> > >> > as
> > >> > > that would be enable us to things like managing deps, CI, Release
> > etc.
> > >> > >
> > >> > > regards,
> > >> > > Rahul
> > >> > >
> > >> > > [1] https://issues.apache.org/jira/browse/HDT-39
> > >> > >
> > >> > >
> > >> > > On Tue, Sep 3, 2013 at 11:26 PM, Adam Berry <adamb@apache.org>
> > wrote:
> > >> > >
> > >> > > > Hi Rahul,
> > >> > > >
> > >> > > > What branch are you trying to do this on?
> > >> > > >
> > >> > > > On master, where Tycho is used, there is a setting in the
pom
> > >> pointing
> > >> > at
> > >> > > > the necessary;
> > >> > > >
> > >> > > >   <repositories>
> > >> > > >     <repository>
> > >> > > >       <id>juno</id>
> > >> > > >       <layout>p2</layout>
> > >> > > >       <url>http://download.eclipse.org/releases/juno</url>
> > >> > > >     </repository>
> > >> > > >   </repositories>
> > >> > > >
> > >> > > > If you are trying to do the same with all local files you
will
> > need
> > >> the
> > >> > > > necessary p2 repositories locally, and the pom configured
to use
> > >> them.
> > >> > I
> > >> > > > think just a copy of the core Eclipse SDK repository and
the JDT
> > one
> > >> > > should
> > >> > > > be sufficient. Then you can just use file: urls to point
to the
> > >> local
> > >> > > > spots.
> > >> > > >
> > >> > > > Or are you trying to not use tycho here and do something
else?
> > >> > > >
> > >> > > > Adam
> > >> > > >
> > >> > > >
> > >> > > > On Tue, Sep 3, 2013 at 10:31 AM, Rahul Sharma <
> > rahul0208@gmail.com>
> > >> > > wrote:
> > >> > > >
> > >> > > > > Hi devs,
> > >> > > > >
> > >> > > > > I tried doing  a maven build this via a simple copy
process,
> > >> where I
> > >> > am
> > >> > > > > copying the required dependencies and generation the
required
> > >> > > packaging.
> > >> > > > > But there are some issues in the same. Eclipse does
not
> publish
> > >> their
> > >> > > > > latest artifacts in maven central or any of the other
> > >> > repositories[1].
> > >> > >  I
> > >> > > > > could only find dependencies for 3.6 version of eclipse
core
> > [2].
> > >> > >  Other
> > >> > > > > related dependencies like swt, Jfaces [2] etc are also
of
> older
> > >> > > versions,
> > >> > > > > available via maven central.
> > >> > > > >
> > >> > > > > In order to get around the problem I checked how they
are
> doing
> > >> it in
> > >> > > > tycho
> > >> > > > > plugin, as they use the latest versions there. They
are
> packing
> > >> their
> > >> > > own
> > >> > > > > jars and we could use a few(osgi, runtime etc), which
can
> > provide
> > >> us
> > >> > > the
> > >> > > > > latest versions. I am not sure of what to with the
rest, if we
> > do
> > >> it
> > >> > in
> > >> > > > > this manner.
> > >> > > > >
> > >> > > > > Much of the problem is due to the eclipse pde
> > >> > > > > libs(core,runtime,osgi,emf,jface,swf etc) that are
required
> for
> > >> the
> > >> > > > build.
> > >> > > > > I have also tried maven-pde plugin but that requires
eclipse
> > path
> > >> of
> > >> > > the
> > >> > > > > machine. I believe it is not actively developed anymore.
> > >> > > > >
> > >> > > > > Are there any suggestions ?
> > >> > > > >
> > >> > > > > regards
> > >> > > > > Rahul
> > >> > > > >
> > >> > > > > [1] https://oss.sonatype.org
> > >> > > > > [2] http://mvnrepository.com/artifact/org.eclipse.core
> > >> > > > > [3] http://mvnrepository.com/artifact/org.eclipse/jface
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message