oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Mattmann <chris.mattm...@gmail.com>
Subject Re: Extra Compiler Tools
Date Wed, 05 Nov 2014 23:42:31 GMT
Mike, have you looked at this yet?

http://techtraits.com/build%20management/maven/2011/09/09/compiling-protoco
l-buffers-from-maven/


I’m going to play with it tonight and see if
I can help here. Do you have some files I can test
with? Can you attach them to JIRA or dropbox them to me
so I can scope?

Cheers,
Chris

------------------------
Chris Mattmann
chris.mattmann@gmail.com




-----Original Message-----
From: Michael Starch <starchmd@umich.edu>
Reply-To: <dev@oodt.apache.org>
Date: Wednesday, November 5, 2014 at 5:37 PM
To: <dev@oodt.apache.org>
Subject: Re: Extra Compiler Tools

>Ok....time for an audible.  Protoc needs to be built from source, no
>binary
>distributions available.  Thus I am going to purge proto-buffers from the
>new code and be done with it.
>
>Any problem making the following classes/interfaces implement
>java.io.Serializable:
>
>JobSpec
>Job
>JobInput
>
>Doing so would allow apache and native java serialization and thus we
>wouldn't need something like proto-buffers.
>
>-Michael
>Thanks Mike +1
>
>------------------------
>Chris Mattmann
>chris.mattmann@gmail.com
>
>
>
>
>-----Original Message-----
>From: Michael Starch <starchmd@umich.edu>
>Reply-To: <dev@oodt.apache.org>
>Date: Wednesday, November 5, 2014 at 12:31 PM
>To: <dev@oodt.apache.org>
>Subject: Re: Extra Compiler Tools
>
>>Looks like you followed the same reasoning chain that I did.  Yes, I came
>>to the same conclusion that ant-build was best.
>>
>>I wasn't sure how to download protoc, but you just answered that....so I
>>think this is a great solution!
>>
>>Thanks,
>>
>>Michael
>>
>>
>>On Wed, Nov 5, 2014 at 10:23 AM, Chris Mattmann
>><chris.mattmann@gmail.com>
>>wrote:
>>
>>> Hi Mike,
>>>
>>> Thanks for flushing this out.
>>>
>>> My thoughts on the below:
>>>
>>>
>>> -----Original Message-----
>>> From: Michael Starch <starchmd@umich.edu>
>>> Reply-To: <dev@oodt.apache.org>
>>> Date: Wednesday, November 5, 2014 at 12:12 PM
>>> To: <dev@oodt.apache.org>
>>> Subject: Re: Extra Compiler Tools
>>>
>>> >I tried this approach. The plugin requires a path to the "protoc" tool
>>>and
>>> >thus a working installation.  This is what prompted the discussion.
>>>
>>> Ah - no worries, what you could do is:
>>>
>>> 1. only enable to plugin if -Pwith-mesos is enabled; and
>>>
>>> >
>>> >Running the plugin under a profile works.
>>>
>>> Yep.
>>>
>>> > However, not running the plugin
>>> >causes compile errors in dependant code.  Excluding this code except
>>> >within
>>> >the profile doesn't seem to work, and is considered by some to be bad
>>>form
>>> >because there is nothing inside the jar file that notes which profiles
>>> >were
>>> >used to compile.
>>>
>>> Got it. Suggestion here would be:
>>>
>>> 2. create a new module, cas-resource-mesos, and inside of that module,
>>> take one of the following approaches, assuming the module is activated
>>> when -Pwith-mesos is enabled:
>>>
>>> 2a. Maven Antrun like so (in this old example):
>>>
>>>http://stackoverflow.com/questions/1578456/integrate-protocol-buffers-in
>>>t
>>>o-
>>> maven2-build
>>>
>>> (pro: more flexibility in case protoc isn¹t there; to fail on error; to
>>> only compile if
>>> protoc is available
>>>
>>> 2b. Maven protobuf plugin
>>> http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html
>>>
>>> Here¹s how to enable a module with a profile:
>>>
>>>
>>>http://blog.soebes.de/blog/2013/11/09/why-is-it-bad-to-activate-slash-de
>>>a
>>>ct
>>> ive-modules-by-profiles-in-maven/
>>>
>>>
>>> It seems like that is a bad idea though, based on that discussion.
>>>
>>> So, here¹s another option:
>>>
>>> 1. Inside of cas-resource (no special new module or anything else)
>>> 2. include some custom Ant magic via a build.xml file and the Maven
>>> AntRun plugin:
>>>   2a. test if protoc is on the system path, and if not, download it,
>>>e.g.,
>>> into the target directory (gets deleted on clean)
>>>   2b. call protoc and compile after 2a
>>>
>>> I would suggest this solution as I think it¹s the most robust and
>>>ensures
>>> we always have a cas-resource that includes mesos and compiled
>>>correctly.
>>>
>>> Cheers,
>>> Chris
>>>
>>> >
>>> >Any ideas on how to continue?
>>> >
>>> >Michael
>>> > On Nov 5, 2014 11:04 AM, "Chris Mattmann" <chris.mattmann@gmail.com>
>>> >wrote:
>>> >
>>> >> Hi Mike,
>>> >>
>>> >> Great discussion. It would be nice if there was
>>> >> a protoc Maven plugin:
>>> >>
>>> >> http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html
>>> >>
>>> >>
>>> >> Looks like there is. My suggestion:
>>> >>
>>> >> 1. use a Profile, something like -Pwith-mesos and
>>> >> then when activated;
>>> >> 2. call the above plugin if -Pwith-mesos is activated
>>> >> in the resource manager
>>> >>
>>> >> Sound good?
>>> >>
>>> >> Cheers,
>>> >> Chris
>>> >>
>>> >> ------------------------
>>> >> Chris Mattmann
>>> >> chris.mattmann@gmail.com
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> -----Original Message-----
>>> >> From: Michael Starch <starchmd@umich.edu>
>>> >> Reply-To: <dev@oodt.apache.org>
>>> >> Date: Wednesday, November 5, 2014 at 11:46 AM
>>> >> To: <dev@oodt.apache.org>
>>> >> Subject: Extra Compiler Tools
>>> >>
>>> >> >All,
>>> >> >
>>> >> >I am trying to integrate apache-mesos with our resource manager.
>>> >>However,
>>> >> >mesos uses a technology called "protobuff" from Google for
>>> >> >marshaling/unmarshaling data.
>>> >> >
>>> >> >This requires running a tool called "protoc" to generate a source
>>>file
>>> >>in
>>> >> >java.  What is the best way to integrate this step into our build
>>> >>process?
>>> >> >
>>> >> >Options I can conceive of:
>>> >> >   -Check in generated java file
>>> >> >   -Require "protoc" installation to build resource manager
>>> >> >   -Separate extra resource package into new module
>>> >> >
>>> >> >None of these ideas are very clean.
>>> >> >
>>> >> >Any other ideas?  I tried setting up a profile to only compile
>>>these
>>> >> >sources when selected, but that turned out not to work.
>>> >> >
>>> >> >-Michael Starch
>>> >>
>>> >>
>>> >>
>>>
>>>
>>>



Mime
View raw message