incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Question about downloading binaries
Date Mon, 02 Apr 2012 19:21:17 GMT
I've created CONNECTORS-443 to describe the current proposal.

Karl

On Mon, Apr 2, 2012 at 3:11 PM, Karl Wright <daddywri@gmail.com> wrote:
> Please see below.
>
> On Mon, Apr 2, 2012 at 3:03 PM, Benson Margulies <bimargulies@gmail.com> wrote:
>> <Jukka added to 'to'; I need backup here so that I don't push you over a cliff.>
>>
>> On Mon, Apr 2, 2012 at 2:19 PM, Karl Wright <daddywri@gmail.com> wrote:
>>> "Since the 3rd-party material is consumed in source code form, I assume
>>> it must have a source-compatible license."
>>>
>>> The releases we patch are:
>>>
>>> - commons-httpclient 3.1
>>> - xerces 2.9.1
>>
>> Uh, oh. Now we have another problem, since these are your fellow
>> Apache people at work.
>>
>> Have you submitted patches back to commons and xerces? What sort of
>> reaction did you get?
>>
>
> I submitted patches for all the materials.  In the case of httpclient,
> the patches were partly accepted and partly rejected - but they are
> only available in the 4.x version of the library, which we have not
> yet gone to.  The reason is complex; the connectors that would use it
> are difficult to test in the absence of available instances of the
> kinds of repositories they would be communicating with.  That's a long
> standing problem I've been trying to find a solution for for more than
> 2 years now.
>
> The patches that were rejected involved things that the package
> developers considered to be "not consistent with mission".  For
> example, the xerces change basically allows the parser to accept
> broken xml (if a certain configuration switch is enabled).
>
>> Releasing 'convenience binaries' of modified sources of Apache
>> projects strikes me as somewhat contrary to the overall goals here.
>> I'd like others to weigh in here, but I'd propose that  you always
>> ship modified Apache products as source. Forget about 'patch'. Just
>> ship modified sources files and drop them into place in fetched copies
>> of their releases, and build the results.
>>
>
> I'd love to acheive closure, believe me, and there are open tickets to
> this end, but until we do it I don't believe it is wise or feasible to
> withhold ManifoldCF from the community.
>
>> As Sebb points out, you really, really, must not push your modified
>> binaries to maven central unless you use shade or equivalent to change
>> the package names.
>>
>
> I would never do that, obviously.
>
>> I wish that others would weigh in here; how bad an idea is a
>> 'convenience binary' consisting of a modified Apache project library?
>>
>>> - jetty 6.1
>>
>> As a courtesy to the Jetty project, the same rule of 'don't stick this
>> out into maven as a standalone artifact' applies. Otherwise, the
>> two-pronged approach is fine: make convenience binaries, and also
>> provide the user with the ability to rebuild them for themselves. I'd
>> propose the 'whole file replacement' mechanism here as well to solve
>> the Windows problem.
>>
>
> Actually it occurred to me that since there are published binary
> releases of these artifacts, I can download and unpack those.  So I
> think I have a solution for this one.
>
> Karl
>
>>>
>>> We also build the jdk1.5 version of hsqldb, which does not require
>>> patching but does require recompilation.
>>
>> That's simple enough as a -dep convenience binary.
>>
>>>
>>> I believe all of these are covered by the Apache 2.0 license.
>>>
>>> Karl
>>>
>>> On Mon, Apr 2, 2012 at 2:14 PM, sebb <sebbaz@gmail.com> wrote:
>>>> On 2 April 2012 17:53, Benson Margulies <bimargulies@gmail.com> wrote:
>>>>> Karl,
>>>>>
>>>>> I hope that you are making this too hard.
>>>>>
>>>>> We don't care about the contents of someone else's binaries. If you
>>>>> make and deploy a -deps  package of third-party binaries, as a
>>>>> convenience for your users, it can contain any strange mixture of
>>>>> sources and binaries it contains. If you provide a script to download
>>>>> a third-party package, we don't care what it downloads so long as the
>>>>> license is acceptable for a dependency.
>>>>>
>>>>> I don't follow the logic that prevents you from having a release manager:
>>>>>
>>>>> a) retrieve third-part sources
>>>>> b) apply your patches from your svn
>>>>> c) create -deps bundle
>>>>> d) deploy to dist, appropriately marked as 'not an Apache release',
>>>>>
>>>>> so long as the third-party material has an acceptable license in the
>>>>> first place.
>>>>
>>>> I would add:
>>>>
>>>> We should not publish patched builds of source in such a way that they
>>>> can interfere with the normal use of the unpatched source builds.
>>>> This includes (but is not limited to) publishing patched binaries on
>>>> Maven Central (regardless of the groupId) with the original package
>>>> names.
>>>>
>>>> Since the 3rd-party material is consumed in source code form, I assume
>>>> it must have a source-compatible license.
>>>> I.e. a license which is only normally permitted for binary
>>>> dependencies would I think be excluded for this use case.
>>>>
>>>>> In case I'm wrong here, I'll ask: what is the build tool of choice for
>>>>> ManifoldCF? On the other hand from all of these, perhaps there is, or
>>>>> could be, a plugin for that build tool that could implement the
>>>>> 'patch' utility for you on Windows?
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Apr 2, 2012 at 12:37 PM, Karl Wright <daddywri@gmail.com>
wrote:
>>>>>> Sorry about the list cc'ing - the gmail client is fighting me today.
>>>>>>
>>>>>> To try and clarify, which will take some time I'm afraid:
>>>>>>
>>>>>> (1) The 0.5 version of the ManifoldCF release candidate was rejected
>>>>>> because the tar contained binary dependencies.  The binary
>>>>>> dependencies were checked into svn.  This has been standard practice
>>>>>> for a decade or more for Java projects built with ant, but has now
>>>>>> been deemed unacceptable.
>>>>>> (2) In trying to find a solution which would still be convenient
but
>>>>>> not include binaries, we considered supplying ant logic that downloads
>>>>>> the dependencies on demand.  The download would use binaries from
the
>>>>>> Maven repository, where possible.
>>>>>> (3) In some cases, there is either no corresponding jar in the Maven
>>>>>> repository, or there is a jar but it doesn't include critical patches.
>>>>>> (4) In these cases, we needed to see whether there was another place
>>>>>> those dependent jars could live.  They were in svn before, so one
>>>>>> possibility was that they remain there.  Other possibilities include
>>>>>> putting them into a googlecode repository, or downloading and building
>>>>>> them as part of the overall build process.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hope this helps.
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 2, 2012 at 12:26 PM, Daniel Shahaf <d.s@daniel.shahaf.name>
wrote:
>>>>>>> Forward to list
>>>>>>>
>>>>>>> I can't answer your question as it's too abstract.  I don't
understand
>>>>>>> what you're trying to do from either infra@ or legal@ perspective.
>>>>>>>
>>>>>>> Karl Wright wrote on Mon, Apr 02, 2012 at 12:21:17 -0400:
>>>>>>>> "'svn patch' exists in 1.7."
>>>>>>>>
>>>>>>>> Ok, so that implies that we would create the patched jar
by checking
>>>>>>>> out the project tag from svn and using svn patch, not by
downloading
>>>>>>>> the source tarball.  Do you think it is ok to allow svn
access as part
>>>>>>>> of a project's build process?
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 2, 2012 at 12:16 PM, Daniel Shahaf <d.s@daniel.shahaf.name>
wrote:
>>>>>>>> > 'svn patch' exists in 1.7.  (and tortoise includes
svn.exe as an
>>>>>>>> > optional component, I hear)
>>>>>>>> >
>>>>>>>> > Karl Wright wrote on Mon, Apr 02, 2012 at 12:05:47 -0400:
>>>>>>>> >> The patches are contained in SVN, yes, but the build
process is
>>>>>>>> >> structured to work on Windows as well as Linux,
and there isn't a
>>>>>>>> >> standard patch utility on Windows.
>>>>>>>> >>
>>>>>>>> >> We could insist that people only build on Linux,
I suppose, but that
>>>>>>>> >> would be a huge inconvenience for a lot of people.
>>>>>>>> >>
>>>>>>>> >> Karl
>>>>>>>> >>
>>>>>>>> >> On Mon, Apr 2, 2012 at 11:47 AM, sebb <sebbaz@gmail.com>
wrote:
>>>>>>>> >> > On 2 April 2012 16:36, Karl Wright <daddywri@gmail.com>
wrote:
>>>>>>>> >> >> ---------- Forwarded message ----------
>>>>>>>> >> >> From: Daniel Shahaf <d.s@daniel.shahaf.name>
>>>>>>>> >> >> Date: Mon, Apr 2, 2012 at 11:35 AM
>>>>>>>> >> >> Subject: Re: Question about downloading
binaries
>>>>>>>> >> >> To: Karl Wright <daddywri@gmail.com>
>>>>>>>> >> >>
>>>>>>>> >> >>
>>>>>>>> >> >> You didn't CC the list
>>>>>>>> >> >>
>>>>>>>> >> >> Karl Wright wrote on Mon, Apr 02, 2012
at 11:33:56 -0400:
>>>>>>>> >> >>> The patches for the dependencies for
the main release are sourced only
>>>>>>>> >> >>> as part of the project in question
at this time.  So there is no other
>>>>>>>> >> >>> logical place for them to live.
>>>>>>>> >> >
>>>>>>>> >> > The project SVN presumably contains the patches?
>>>>>>>> >> > If not, it should.
>>>>>>>> >> >
>>>>>>>> >> > In which case, can't you download the source
and apply the patches as
>>>>>>>> >> > part of the build process?
>>>>>>>> >> >
>>>>>>>> >> > This is what the Tomcat project does (though
it's not changing code,
>>>>>>>> >> > just changing package names to avoid collisions).
>>>>>>>> >> >
>>>>>>>> >> >>> Karl
>>>>>>>> >> >>>
>>>>>>>> >> >>> On Mon, Apr 2, 2012 at 11:31 AM, Daniel
Shahaf <d.s@daniel.shahaf.name> wrote:
>>>>>>>> >> >>> > Why do they have to be hosted
on Apache infrastructure?  The Subversion
>>>>>>>> >> >>> > build depends on a C compiler
but we don't host that on ASF hardware.
>>>>>>>> >> >>> >
>>>>>>>> >> >>> > Karl Wright wrote on Mon, Apr
02, 2012 at 11:16:22 -0400:
>>>>>>>> >> >>> >> Hi folks,
>>>>>>>> >> >>> >>
>>>>>>>> >> >>> >> As a result of a change in
the way Java releases must be structured,
>>>>>>>> >> >>> >> we need to be able to download
binary dependencies as part of the
>>>>>>>> >> >>> >> build process.  The problem
is that we have some binary dependencies
>>>>>>>> >> >>> >> that contain patches, or are
otherwise unsuitable for being pushed to
>>>>>>>> >> >>> >> the Maven repository.  What
is the best place in the Apache
>>>>>>>> >> >>> >> infrastructure for keeping
dependencies like this?
>>>>>>>> >> >>> >>
>>>>>>>> >> >>> >> Thanks,
>>>>>>>> >> >>> >> Karl
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message