incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: Question about downloading binaries
Date Mon, 02 Apr 2012 19:39:05 GMT
On 2 April 2012 20:11, Karl Wright <daddywri@gmail.com> wrote:
> Please see below.
>
> On Mon, Apr 2, 2012 at 3:03 PM, Benson Margulies <bimargulies@gmail.com> wrote:
>> <Jukka added to 'to'; I need backup here so that I don't push you over a cliff.>
>>
>> On Mon, Apr 2, 2012 at 2:19 PM, Karl Wright <daddywri@gmail.com> wrote:
>>> "Since the 3rd-party material is consumed in source code form, I assume
>>> it must have a source-compatible license."
>>>
>>> The releases we patch are:
>>>
>>> - commons-httpclient 3.1
>>> - xerces 2.9.1
>>
>> Uh, oh. Now we have another problem, since these are your fellow
>> Apache people at work.
>>
>> Have you submitted patches back to commons and xerces? What sort of
>> reaction did you get?
>>
>
> I submitted patches for all the materials.  In the case of httpclient,
> the patches were partly accepted and partly rejected - but they are
> only available in the 4.x version of the library, which we have not
> yet gone to.

You really should upgrade; 3.x is end-of-life.

> The reason is complex; the connectors that would use it
> are difficult to test in the absence of available instances of the
> kinds of repositories they would be communicating with.  That's a long
> standing problem I've been trying to find a solution for for more than
> 2 years now.

Depending on the original source, it may be possible to provide local
code that works round the issues.
This can generally be in your own package space.

If the required hooks are missing in the libraries, have you tried
asking the developers to provide a hook?

That may be possible without compromising the original library.

> The patches that were rejected involved things that the package
> developers considered to be "not consistent with mission".  For
> example, the xerces change basically allows the parser to accept
> broken xml (if a certain configuration switch is enabled).

I'm not surprised that was rejected.
There has to be a better way to do that.

>> Releasing 'convenience binaries' of modified sources of Apache
>> projects strikes me as somewhat contrary to the overall goals here.
>> I'd like others to weigh in here, but I'd propose that  you always
>> ship modified Apache products as source. Forget about 'patch'. Just
>> ship modified sources files and drop them into place in fetched copies
>> of their releases, and build the results.
>>
>
> I'd love to acheive closure, believe me, and there are open tickets to
> this end, but until we do it I don't believe it is wise or feasible to
> withhold ManifoldCF from the community.
>
>> As Sebb points out, you really, really, must not push your modified
>> binaries to maven central unless you use shade or equivalent to change
>> the package names.
>>
>
> I would never do that, obviously.
>
>> I wish that others would weigh in here; how bad an idea is a
>> 'convenience binary' consisting of a modified Apache project library?
>>
>>> - jetty 6.1
>>
>> As a courtesy to the Jetty project, the same rule of 'don't stick this
>> out into maven as a standalone artifact' applies. Otherwise, the
>> two-pronged approach is fine: make convenience binaries, and also
>> provide the user with the ability to rebuild them for themselves. I'd
>> propose the 'whole file replacement' mechanism here as well to solve
>> the Windows problem.
>>
>
> Actually it occurred to me that since there are published binary
> releases of these artifacts, I can download and unpack those.  So I
> think I have a solution for this one.
>
> Karl
>
>>>
>>> We also build the jdk1.5 version of hsqldb, which does not require
>>> patching but does require recompilation.
>>
>> That's simple enough as a -dep convenience binary.
>>
>>>
>>> I believe all of these are covered by the Apache 2.0 license.
>>>
>>> Karl
>>>
>>> On Mon, Apr 2, 2012 at 2:14 PM, sebb <sebbaz@gmail.com> wrote:
>>>> On 2 April 2012 17:53, Benson Margulies <bimargulies@gmail.com> wrote:
>>>>> Karl,
>>>>>
>>>>> I hope that you are making this too hard.
>>>>>
>>>>> We don't care about the contents of someone else's binaries. If you
>>>>> make and deploy a -deps  package of third-party binaries, as a
>>>>> convenience for your users, it can contain any strange mixture of
>>>>> sources and binaries it contains. If you provide a script to download
>>>>> a third-party package, we don't care what it downloads so long as the
>>>>> license is acceptable for a dependency.
>>>>>
>>>>> I don't follow the logic that prevents you from having a release manager:
>>>>>
>>>>> a) retrieve third-part sources
>>>>> b) apply your patches from your svn
>>>>> c) create -deps bundle
>>>>> d) deploy to dist, appropriately marked as 'not an Apache release',
>>>>>
>>>>> so long as the third-party material has an acceptable license in the
>>>>> first place.
>>>>
>>>> I would add:
>>>>
>>>> We should not publish patched builds of source in such a way that they
>>>> can interfere with the normal use of the unpatched source builds.
>>>> This includes (but is not limited to) publishing patched binaries on
>>>> Maven Central (regardless of the groupId) with the original package
>>>> names.
>>>>
>>>> Since the 3rd-party material is consumed in source code form, I assume
>>>> it must have a source-compatible license.
>>>> I.e. a license which is only normally permitted for binary
>>>> dependencies would I think be excluded for this use case.
>>>>
>>>>> In case I'm wrong here, I'll ask: what is the build tool of choice for
>>>>> ManifoldCF? On the other hand from all of these, perhaps there is, or
>>>>> could be, a plugin for that build tool that could implement the
>>>>> 'patch' utility for you on Windows?
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Apr 2, 2012 at 12:37 PM, Karl Wright <daddywri@gmail.com>
wrote:
>>>>>> Sorry about the list cc'ing - the gmail client is fighting me today.
>>>>>>
>>>>>> To try and clarify, which will take some time I'm afraid:
>>>>>>
>>>>>> (1) The 0.5 version of the ManifoldCF release candidate was rejected
>>>>>> because the tar contained binary dependencies.  The binary
>>>>>> dependencies were checked into svn.  This has been standard practice
>>>>>> for a decade or more for Java projects built with ant, but has now
>>>>>> been deemed unacceptable.
>>>>>> (2) In trying to find a solution which would still be convenient
but
>>>>>> not include binaries, we considered supplying ant logic that downloads
>>>>>> the dependencies on demand.  The download would use binaries from
the
>>>>>> Maven repository, where possible.
>>>>>> (3) In some cases, there is either no corresponding jar in the Maven
>>>>>> repository, or there is a jar but it doesn't include critical patches.
>>>>>> (4) In these cases, we needed to see whether there was another place
>>>>>> those dependent jars could live.  They were in svn before, so one
>>>>>> possibility was that they remain there.  Other possibilities include
>>>>>> putting them into a googlecode repository, or downloading and building
>>>>>> them as part of the overall build process.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hope this helps.
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 2, 2012 at 12:26 PM, Daniel Shahaf <d.s@daniel.shahaf.name>
wrote:
>>>>>>> Forward to list
>>>>>>>
>>>>>>> I can't answer your question as it's too abstract.  I don't
understand
>>>>>>> what you're trying to do from either infra@ or legal@ perspective.
>>>>>>>
>>>>>>> Karl Wright wrote on Mon, Apr 02, 2012 at 12:21:17 -0400:
>>>>>>>> "'svn patch' exists in 1.7."
>>>>>>>>
>>>>>>>> Ok, so that implies that we would create the patched jar
by checking
>>>>>>>> out the project tag from svn and using svn patch, not by
downloading
>>>>>>>> the source tarball.  Do you think it is ok to allow svn
access as part
>>>>>>>> of a project's build process?
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 2, 2012 at 12:16 PM, Daniel Shahaf <d.s@daniel.shahaf.name>
wrote:
>>>>>>>> > 'svn patch' exists in 1.7.  (and tortoise includes
svn.exe as an
>>>>>>>> > optional component, I hear)
>>>>>>>> >
>>>>>>>> > Karl Wright wrote on Mon, Apr 02, 2012 at 12:05:47 -0400:
>>>>>>>> >> The patches are contained in SVN, yes, but the build
process is
>>>>>>>> >> structured to work on Windows as well as Linux,
and there isn't a
>>>>>>>> >> standard patch utility on Windows.
>>>>>>>> >>
>>>>>>>> >> We could insist that people only build on Linux,
I suppose, but that
>>>>>>>> >> would be a huge inconvenience for a lot of people.
>>>>>>>> >>
>>>>>>>> >> Karl
>>>>>>>> >>
>>>>>>>> >> On Mon, Apr 2, 2012 at 11:47 AM, sebb <sebbaz@gmail.com>
wrote:
>>>>>>>> >> > On 2 April 2012 16:36, Karl Wright <daddywri@gmail.com>
wrote:
>>>>>>>> >> >> ---------- Forwarded message ----------
>>>>>>>> >> >> From: Daniel Shahaf <d.s@daniel.shahaf.name>
>>>>>>>> >> >> Date: Mon, Apr 2, 2012 at 11:35 AM
>>>>>>>> >> >> Subject: Re: Question about downloading
binaries
>>>>>>>> >> >> To: Karl Wright <daddywri@gmail.com>
>>>>>>>> >> >>
>>>>>>>> >> >>
>>>>>>>> >> >> You didn't CC the list
>>>>>>>> >> >>
>>>>>>>> >> >> Karl Wright wrote on Mon, Apr 02, 2012
at 11:33:56 -0400:
>>>>>>>> >> >>> The patches for the dependencies for
the main release are sourced only
>>>>>>>> >> >>> as part of the project in question
at this time.  So there is no other
>>>>>>>> >> >>> logical place for them to live.
>>>>>>>> >> >
>>>>>>>> >> > The project SVN presumably contains the patches?
>>>>>>>> >> > If not, it should.
>>>>>>>> >> >
>>>>>>>> >> > In which case, can't you download the source
and apply the patches as
>>>>>>>> >> > part of the build process?
>>>>>>>> >> >
>>>>>>>> >> > This is what the Tomcat project does (though
it's not changing code,
>>>>>>>> >> > just changing package names to avoid collisions).
>>>>>>>> >> >
>>>>>>>> >> >>> Karl
>>>>>>>> >> >>>
>>>>>>>> >> >>> On Mon, Apr 2, 2012 at 11:31 AM, Daniel
Shahaf <d.s@daniel.shahaf.name> wrote:
>>>>>>>> >> >>> > Why do they have to be hosted
on Apache infrastructure?  The Subversion
>>>>>>>> >> >>> > build depends on a C compiler
but we don't host that on ASF hardware.
>>>>>>>> >> >>> >
>>>>>>>> >> >>> > Karl Wright wrote on Mon, Apr
02, 2012 at 11:16:22 -0400:
>>>>>>>> >> >>> >> Hi folks,
>>>>>>>> >> >>> >>
>>>>>>>> >> >>> >> As a result of a change in
the way Java releases must be structured,
>>>>>>>> >> >>> >> we need to be able to download
binary dependencies as part of the
>>>>>>>> >> >>> >> build process.  The problem
is that we have some binary dependencies
>>>>>>>> >> >>> >> that contain patches, or are
otherwise unsuitable for being pushed to
>>>>>>>> >> >>> >> the Maven repository.  What
is the best place in the Apache
>>>>>>>> >> >>> >> infrastructure for keeping
dependencies like this?
>>>>>>>> >> >>> >>
>>>>>>>> >> >>> >> Thanks,
>>>>>>>> >> >>> >> Karl
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message