hdt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srimanth Gunturi <srima...@gmail.com>
Subject Re: Maven build
Date Thu, 26 Sep 2013 08:36:47 GMT
Hello,
Requesting review and commit of patch attached to
https://issues.apache.org/jira/browse/HDT-40.
Was able to connect to ZK and HDFS services, and browse as expected.
Would be great if others could verify it works for them too.
Regards,
Srimanth





On Tue, Sep 17, 2013 at 12:39 AM, Srimanth Gunturi <srimanth@gmail.com>wrote:

> Hello,
> With Rahul's excellent work in getting maven build issues resolved, we
> will be able to get back to feature development.
> A good release point for the project would be to have all the features of
> HDT and Hadoop-Eclipse projects working.
>
> The big missing piece of this merge is the existing MapReduce
> functionality of HDT.
> I have created https://issues.apache.org/jira/browse/HDT-41 to address
> this issue.
> I would like to request assignment of HDT-41 and HDT-40 to me, as I have
> started work on it.
>
> I will upload patches as soon as I have them locally working.
>
> On a side note, I would like updating of JIRA's components to include
> 'common', 'zookeeper', 'core' and 'ui'.
> Regards,
> Srimanth
>
>
>
>
>
>
> On Tue, Sep 10, 2013 at 1:58 AM, Rahul Sharma <rahul0208@gmail.com> wrote:
>
>> Hi Adam,
>>
>> I have made a fix for the issue at hand using tycho, bundle-classpath and
>> copy-dependencies. Basically during a maven build I am copying the
>> required
>> dependencies into a folder. These dependencies are  referenced in
>> MANIFEST.MF using Bundle-classpath. The copying takes place before tycho
>> kicks in and thus these are available for  the compiler. This relieves us
>> from creation of multiple modules which need to be triggered in multiple
>> profiles. Also we  do not need to create any separate bundles as the
>> required libs are bundles with-in the plugin. In turn we are quite
>> portable, without sacrificing any configuration like MANIFEST,classpath or
>> any of the ecllipse settings.
>>
>> I have uploaded  the updated patch in HDT-39[1], have a look at it and let
>> me know your thoughts for the same.
>>
>> regards
>> Rahul
>>
>> [1]
>>
>> https://issues.apache.org/jira/secure/attachment/12602309/0001-HDT-39-Using-maven-to-build-package.patch
>>
>>
>>
>> On Tue, Sep 10, 2013 at 8:41 AM, Adam Berry <adamb@apache.org> wrote:
>>
>> > Hi Rahul,
>> >
>> > So I've been thinking about this. I know that we cannot build the
>> > dependencies wrapper and the IDE plugins in the same reactor build,
>> which
>> > rules out the multi module approach. I looked at the current patch is
>> up,
>> > which makes use of profiles to separate this, as profiles are about
>> build
>> > portability, I'm not sure thats quite right.
>> >
>> > Multi module really represents how this should be done, but as we can't
>> use
>> > it I propose that we actually make a new maven project to build the
>> > dependency wrapper plugins, and then document that to build the plugins
>> > requires two maven commands (at this time). Building HDT can reference
>> a p2
>> > repository output from the dependency build, and this can take a local
>> > path.
>> >
>> > I will work up a path with this approach, and add a comment on HDT-39 to
>> > this effect, and we'll see if everyone is ok with this.
>> >
>> > Cheers,
>> > Adam
>> >
>> >
>> > On Tue, Sep 3, 2013 at 10:08 PM, Rahul Sharma <rahul0208@gmail.com>
>> wrote:
>> >
>> > > Hi Adam,
>> > >
>> > > This is in relation to issue HDT-39[1] of hadoop-eclipse -merging. I
>> > > configured things using tycho but Srimanth had a different idea of
>> > things.
>> > > This is due to the different approach he took for hadoop-eclipse. I
>> still
>> > > think the same can be achieved done using tycho. There is nothing
>> > stopping
>> > > us to create specific bundles  for different versions of hadoop,
>> which is
>> > > being done at other Apache projects like servicemix, provisonr etc.
>> > >
>> > > Anyways, the idea was to use maven as dumb possible, just for
>> > dependencies
>> > > management, and not to use any of tricks of the trade. I tried to dig
>> on
>> > > the possible alternatives but I do not think besides tycho there is
>> any
>> > > other cleaner approach. In all the rest of the approaches we have to
>> keep
>> > > eclipse SDK. Apparently maven and eclipse are two separate worlds
>> that do
>> > > not converge easily.
>> > >
>> > > Let me know thoughts for the same as I think we would like to use
>> maven
>> > as
>> > > that would be enable us to things like managing deps, CI, Release etc.
>> > >
>> > > regards,
>> > > Rahul
>> > >
>> > > [1] https://issues.apache.org/jira/browse/HDT-39
>> > >
>> > >
>> > > On Tue, Sep 3, 2013 at 11:26 PM, Adam Berry <adamb@apache.org> wrote:
>> > >
>> > > > Hi Rahul,
>> > > >
>> > > > What branch are you trying to do this on?
>> > > >
>> > > > On master, where Tycho is used, there is a setting in the pom
>> pointing
>> > at
>> > > > the necessary;
>> > > >
>> > > >   <repositories>
>> > > >     <repository>
>> > > >       <id>juno</id>
>> > > >       <layout>p2</layout>
>> > > >       <url>http://download.eclipse.org/releases/juno</url>
>> > > >     </repository>
>> > > >   </repositories>
>> > > >
>> > > > If you are trying to do the same with all local files you will need
>> the
>> > > > necessary p2 repositories locally, and the pom configured to use
>> them.
>> > I
>> > > > think just a copy of the core Eclipse SDK repository and the JDT one
>> > > should
>> > > > be sufficient. Then you can just use file: urls to point to the
>> local
>> > > > spots.
>> > > >
>> > > > Or are you trying to not use tycho here and do something else?
>> > > >
>> > > > Adam
>> > > >
>> > > >
>> > > > On Tue, Sep 3, 2013 at 10:31 AM, Rahul Sharma <rahul0208@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Hi devs,
>> > > > >
>> > > > > I tried doing  a maven build this via a simple copy process,
>> where I
>> > am
>> > > > > copying the required dependencies and generation the required
>> > > packaging.
>> > > > > But there are some issues in the same. Eclipse does not publish
>> their
>> > > > > latest artifacts in maven central or any of the other
>> > repositories[1].
>> > >  I
>> > > > > could only find dependencies for 3.6 version of eclipse core
[2].
>> > >  Other
>> > > > > related dependencies like swt, Jfaces [2] etc are also of older
>> > > versions,
>> > > > > available via maven central.
>> > > > >
>> > > > > In order to get around the problem I checked how they are doing
>> it in
>> > > > tycho
>> > > > > plugin, as they use the latest versions there. They are packing
>> their
>> > > own
>> > > > > jars and we could use a few(osgi, runtime etc), which can provide
>> us
>> > > the
>> > > > > latest versions. I am not sure of what to with the rest, if we
do
>> it
>> > in
>> > > > > this manner.
>> > > > >
>> > > > > Much of the problem is due to the eclipse pde
>> > > > > libs(core,runtime,osgi,emf,jface,swf etc) that are required for
>> the
>> > > > build.
>> > > > > I have also tried maven-pde plugin but that requires eclipse
path
>> of
>> > > the
>> > > > > machine. I believe it is not actively developed anymore.
>> > > > >
>> > > > > Are there any suggestions ?
>> > > > >
>> > > > > regards
>> > > > > Rahul
>> > > > >
>> > > > > [1] https://oss.sonatype.org
>> > > > > [2] http://mvnrepository.com/artifact/org.eclipse.core
>> > > > > [3] http://mvnrepository.com/artifact/org.eclipse/jface
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message