river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter <j...@zeus.net.au>
Subject Re: Build system and Java 8 was: Re: [Vote] (RIVER-432) Jar files in svn and src distributions
Date Wed, 12 Feb 2014 05:22:43 GMT
Well, River doesn't build, throws exceptions, it won't work for client code either and it doesn't
support Java 8 features.

I wrote the java 5 language support code, I would rather spend time reorganising the build
once, than maintain a build tool. 

Just my thoughts.

----- Original message -----
> 
> Hi Peter:
> 
> I’m not familiar with Java 8 yet.   How is the current build not
> supported?
> 
> Cheers,
> 
> Greg.
> 
> On Feb 11, 2014, at 5:35 AM, Peter <jini@zeus.net.au> wrote:
> 
> > +1 Peter.
> > 
> > Note that the current build system does not support java 8.     
> > 
> > Maintaining a build system is a significant burden ( I know I had to
> > implement Java 5 support).
> > 
> > We will need a new build system for java 8, this looks like a step in
> > that direction.   In reality we need to adopt the build conventions
> > used by Rio.
> > 
> > Regards,
> > 
> > Peter.
> > ----- Original message -----
> > > 
> > > As discussion has settled somewhat, I would like to call another
> > > vote to accept the latest patch described in 
> > > https://issues.apache.org/jira/browse/RIVER-432
> > > 
> > > The patch removes the archived jar files for Velocity and ASM and
> > > replaces them with Apache Ivy scripts that download the jars from
> > > Maven Central the first time a build is done.     From then on, the
> > > jar files are in Ivy’s repository (for more info, see
> > > http://ant.apache.org/ivy).
> > > 
> > > Voting will remain open at least until 2000 UTC Feb 13, 2014.
> > > 
> > > Cheers,
> > > 
> > > Greg.
> > > 
> > > 
> > > On Jan 3, 2014, at 12:57 PM, Greg Trasuk <trasukg@stratuscom.com>
> > > wrote:
> > > 
> > > > 
> > > > On Jan 3, 2014, at 5:25 AM, Simon IJskes - QCG <simon@qcg.nl>
> > > > wrote:
> > > > 
> > > > > In order to gain some time to discuss this first i will vote -1.
> > > > > 
> > > > > First, we decided to NOT remove velocity builder.
> > > > 
> > > > When I read the email chain, my impression was that we wanted to
> > > > remove it (to quote you Sim, “To be honest, I hate it”), but there
> > > > was a dependency on it in the ‘extras’ folder that was added in
> > > > the trunk branch.     As there is no ‘extras’ in the 2.2 branch,
and
> > > > that is what this patch applies to, I thought it was fair to remove
> > > > VelocityConfigurationBuilder from the 2.2 branch.         Perhaps
we
> > > > should revisit the ConfigurationBuilder approach in another
> > > > thread.     For now I’ll spin another patch that doesn’t remove
> > > > VelocityConfigurationBuilder.
> > > > 
> > > > > 
> > > > > Second, no need to remove the jars as specified in your own
> > > > > comments on RIVER-432.
> > > > > 
> > > > > Pulling in external jars at compile time, shall we start here?
> > > > > 
> > > > > They are already in the svn. They are already in the build
> > > > > scripts. What does this patch fix? No legal problems?
> > > > > 
> > > > 
> > > > Apache policy is somewhat unclear on this point.     One needs to
> > > > examine the mailing lists for clues on what we should really do.   
> > > > I will argue that:
> > > > 
> > > > 1 - The fundamental distribution model of Apache is source code,
> > > > not binaries. 2 - Distributing binaries is tolerated but not
> > > > encouraged.   Since the svn repository can be seen as a
> > > > distribution point, binaries in svn are also tolerated but not
> > > > encouraged. 3 - Downloading dependency binaries at build time is
> > > > technologically easy, provides the same guarantees as putting them
> > > > in cvs, and avoids the question of effectively distributing
> > > > someone else’s code.
> > > > 
> > > > All these together suggest that although we’re technically OK to
> > > > put dependency jars in a “-deps” package (note that the status quo
> > > > _is_ unacceptable - at the very least, we need to restructure the
> > > > dependencies into a “-deps” binary package), there is some policy
> > > > uncertainty which we avoid totally by having dependencies
> > > > downloaded from a known-good source at build time.
> > > > 
> > > > Let’s examine these points:
> > > > 
> > > > 1 - The fundamental distribution model of Apache is source code,
> > > > not binaries.     Apache began with httpd.     Back in those days,
> > > > “Open Source” software was distributed in source form only, simply
> > > > because it was mostly intended for Unix systems (then later
> > > > Linux).     I recall the first release of Perl coming down as a
> > > > ten-part uunet news message.   Part of this distribution model was
> > > > practical necessity - System differences made it necessary to
> > > > compile your software on the hardware it was going to run on.   
> > > > Part of it was open-source philosophy.   Having the source code
> > > > meant that you could take a look at it and verify that it wasn’t
> > > > hazardous to your operations.     
> > > > 
> > > > In any case, the way we use to use open source software was
> > > > (“./configure; make; make install”).     If the software had
> > > > dependencies, you built them from source, for the same reasons.
> > > > 
> > > > Now, as time has gone on, we’ve gotten used to having the JVM as a
> > > > common runtime, and jar files as a common binary distribution
> > > > medium.   But the Apache Foundation’s mandate is to produce open
> > > > source software that is freely usable under the Apache License.   
> > > > That means source code is at the heart of Apache, despite the rest
> > > > of the world’s comfort with binaries.     Hence Roy’s statements
in
> > > > (1):
> > > > 
> > > > > Class files are not open source.     Jar files filled with class
> > > > > files are not open source.     The fact that they are derived from
> > > > > open source is applicable only to what we allow projects to be
> > > > > dependent upon, not what we vote on as a release package.   
> > > > > Release votes are on verified open source artifacts.     Binary
> > > > > packages are separate from source packages. One cannot vote to
> > > > > approve a release containing a mix of source and binary code
> > > > > because the binary is not open source and cannot be verified to
> > > > > be safe for release (even if it was derived from open source).
> > > > > 
> > > > > I thought that was frigging obvious.     Why do I need to write
> > > > > documentation to explain something that is fundamental to the
> > > > > open source definition?
> > > > He’s talking about binary packages, not jar files in svn, but I
> > > > read that (and many other emails) as a distaste for binary
> > > > distributions.
> > > > 
> > > > In fact, if you look at Apache httpd’s download page, it doesn’t
> > > > appear that the Apache project publishes any Unix or Linux
> > > > binaries.   They leave that to other organizations.
> > > > 
> > > > 2 - Distributing binaries is tolerated but not encouraged.     Since
> > > > the svn repository can be seen as a distribution point, binaries
> > > > in svn are also tolerated but not encouraged.
> > > > 
> > > > It’s hard to find a single reference that encapsulates this
> > > > outlook, but that’s the impression I get from reading the various
> > > > mailing lists.     For instance, Sam Ruby says (2):
> > > > > IMO, our projects release source. So, our projects should not
> > > > > maintain object/binary artifacts in their svn release tree,
> > > > > regardless of license (category a or b).
> > > > There is some debate on whether the svn tree should be considered a
> > > > distribution point.     Incubator releases are regularly called out
> > > > for not having “NOTICE” and “RELEASE” files at all reasonable
> > > > checkout points in svn.     [LEGAL-26]
> > > > (https://issues.apache.org/jira/browse/LEGAL-26) concerns this and
> > > > remains unresolved.
> > > > 
> > > > Doug Cutting (3) says:
> > > > > On Mon, Sep 16, 2013 at 2:50 AM, Stephen Connolly
> > > > > <stephen.alan.connolly@gmail.com> wrote:
> > > > > > * Source control is not an Apache distribution and hence we
do
> > > > > > not need to have LICENSE and NOTICE files in source control,
> > > > > > it can be a nice convenience, but there is no *requirement*.
> > > > > 
> > > > > I think perhaps you're looking for clear lines where things are
> > > > > actually a bit fuzzy.     Certainly releases are official
> > > > > distributions and need LICENSE and NOTICE files.     That line
is
> > > > > clear.     On the other hand, we try to discourage folks from
> > > > > thinking that source control is a distribution.     Rather we wish
> > > > > it to be considered our shared workspace, containing works in
> > > > > progress, not yet always ready for distribution to folks outside
> > > > > the foundation.     But, since we work in public, folks from
> > > > > outside the foundation can see our shared workspace and might
> > > > > occasionally mistake it for an official distribution.     We'd
> > > > > like them to still see a LICENSE and NOTICE file.     So it's not
> > > > > a hard-and-fast requirement that every tree that can possibly be
> > > > > checked out have a LICENSE and NOTICE file at its root, but it's
> > > > > a good practice for those trees that are likely to be checked
> > > > > out have them, so that folks who might consume them are well
> > > > > informed.
> > > > Again, he’s not talking directly about jar files in svn, however I
> > > > think his statement that “since we work in public, folks from
> > > > outside the foundation can see our shared workspace and might
> > > > occasionally mistake it for an official distribution” applies
> > > > here.     Fundamentally, the decision on how and where to distribute
> > > > ‘velocity.jar’ rightly belongs with the Velocity group and I don’t
> > > > think we ought to redistribute it.
> > > > 
> > > > 3 - Downloading dependency binaries at build time is
> > > > technologically easy, provides the same guarantees as putting them
> > > > in cvs, and avoids the question of effectively distributing
> > > > someone else’s code.
> > > > 
> > > > There doesn’t seem to be clear policy in the ASF on this, as
> > > > evidenced by the frequent debates on it, and the lack of
> > > > documentation.     I’ve tried to lay out an argument that having
> > > > jars in svn is not encouraged by the ASF (really, it’s not in line
> > > > with the ASF’s charter), even if it isn’t disallowed.     You may
> > > > disagree, and I won’t claim I’ve made a strong argument, simply
> > > > because the policy isn’t clear.     So instead of going through
> > > > arguments that amount to differences of opinion on Apache policy,
> > > > let’s use a technological solution that is simple, common, and
> > > > avoids the question altogether, by automatically downloading the
> > > > dependencies at build time.
> > > > 
> > > > Projects that use Maven do this automatic download as standard
> > > > practice (that’s what Maven does, and that’s what the Maven Central
> > > > infrastructure is there to support).     We don’t use Maven, which
is
> > > > fine (our customers have asked us to make our binaries available in
> > > > Maven Central, and we’ve done that).     Apache Ivy is a popular
> > > > add-on to Apache Ant that provides similar dependency resolution
> > > > to an Ant-based build.
> > > > 
> > > > I was a little surprised how easy it was to persuade Ivy to get the
> > > > required dependencies at build time.     The “ivy.xml” file is 39
> > > > lines including the ASL header (which by the way I forgot to
> > > > include in the patch - I’ll fix that).     There are about 50 lines
> > > > added to ‘build.xml’ to download Ivy and then download the
> > > > required jar files
> > > > 
> > > > So, given that the status-quo seems to be unacceptable (Roy talks
> > > > about not having jar files in the open-source trees, only in
> > > > “-deps” and “tools” trees), we have two options:
> > > > 
> > > > (a) - restructure the svn repository and the build to allow a
> > > > separate “-deps” distribution.     This wouldn’t affect our binary
> > > > distributions (note that I’m no longer using the term “binary
> > > > release”), but to build from source, a user would have to download
> > > > a separate archive, unpack it, and then copy those files into the
> > > > directory that was unpacked from the source release.     This option
> > > > effectively still has us distributing dependent binaries, which is
> > > > not the goal of the ASF, just with a disclaimer that says “this
> > > > isn’t an ASF release, its just a binary distribution put together
> > > > by a committer for your convenience, so don’t feel you should
> > > > trust it too much”.
> > > > 
> > > > (b) - use Ivy to get the jars from Maven Central automatically as
> > > > part of the build.
> > > > 
> > > > I think (b) is the option that causes the least hassle for our
> > > > downstream consumers, and not much hassle for us.
> > > > 
> > > > 
> > > > > Pulling external jars at compile time also makes it more
> > > > > difficult to certify the software. In order to certify the
> > > > > software you need to establish baseline that will be garanteed
> > > > > the same, even if you pull it from the archive 10 years later.
> > > > 
> > > > As I said above, Apache’s focus is creating software that can be
> > > > built from source, not distributing binaries (note that QCG or
> > > > another company might have a different focus, and is perfectly
> > > > able to distribute binaries under the Apache license).     I think a
> > > > reasonably prudent user would ask “How can I trust the
> > > > ‘velocity.jar’ that’s included in this binary?”     And the
answer
> > > > would be either “because I built it from source and installed it
> > > > in my corporate repository” (very cautious, but not unheard-of) or
> > > > “It was published by the Velocity group to a trusted repository,
> > > > Maven Central” (more common).
> > > > 
> > > > If you look in the “ivy.xml” file you’ll see that the dependencies
> > > > are specified using Maven-style “group-artifact-version”
> > > > coordinates.     Old versions are maintained in Maven Central
> > > > forever.     I suppose it’s possible that a publisher could convince
> > > > Maven Central to remove a version for some reason (security or
> > > > licensing problems perhaps), but then, would we want to be
> > > > distributing that version in a “-deps” package?
> > > > 
> > > > I agree that it’s not enough to just say “you need to download
> > > > such-and-such jar”, hence the automatic download managed by “Ivy”
> > > > from Maven Central.
> > > > 
> > > > > It is not a high level project that builds on several
> > > > > frameworks. It is a lowlevel system library. The stuff below the
> > > > > stack is minimal. The number of jars we use is limited. Why
> > > > > bother?
> > > > > 
> > > > 
> > > > In the currently released branches, the dependencies are limited to
> > > > ASM and Velocity.     Looking forward to the trunk branch and the
> > > > qa_refactor branch, the number of external dependencies seem to be
> > > > increasing (IMO I don’t like that, because I also view River as a
> > > > low level system library, but I’m only one PMC member).     We need
> > > > to get in front of the problem before we start distributing large
> > > > numbers of dependencies.
> > > > 
> > > > This point rolls in with the general question of jar files in
> > > > version control.     I was always taught that version control was
> > > > for source code, and putting binaries into version control was a
> > > > bad idea.     In addition, there are practical problems - with older
> > > > systems like cvs, even doing an update or commit effectively
> > > > downloads the binaries, which slows things down if there are large
> > > > binary files.     On newer distributed version control systems like
> > > > git or Mercurial, the entire repository, including all versions of
> > > > binary artifacts, comes down with the project checkout.   
> > > > Currently, we have one version of relatively few jar files in our
> > > > repository, so it’s not a major issue. But it gets worse as time
> > > > goes on.     So I suggest we work out the technology now to avoid
> > > > the problem.
> > > > 
> > > > > Gr. Simon
> > > > > 
> > > > 
> > > > Thanks for the questions, Sim.     I hope you’ll come around to
> > > > removing your ‘-1’.
> > > > 
> > > > Cheers,
> > > > 
> > > > Greg
> > > > 
> > > > Footnotes
> > > > ——————
> > > > 
> > > > (1) - Roy Fielding - http://s.apache.org/roy-binary-deps-1
> > > > (2) - Sam Ruby - http://s.apache.org/r5
> > > > (3) - Doug Cutting - http://s.apache.org/GNP
> > > > 
> > > > > On 02-01-14 18:22, Greg Trasuk wrote:
> > > > > > 
> > > > > > Hello all:
> > > > > > 
> > > > > > Please have a look at the patch mentioned below and cast a
> > > > > > vote on it.
> > > > > > 
> > > > > > The main idea is to remove the dependency jar files from the
> > > > > > source distribution.     As a side effect of using Ivy, it
> > > > > > becomes reasonable to remove them from the svn archive as
> > > > > > well.     Also, the Velocity dependency was there to support
the
> > > > > > VelocityConfigurationBuilder.     We had discussed removing
that
> > > > > > component, so rather than move that dependency to Ivy, I’ve
> > > > > > removed VelocityConfigurationBuilder.
> > > > > > 
> > > > > > It’s arguable whether the VelocityConfigurationBuider was
part
> > > > > > of the official Jini API (I see it as a utility, not API), so
> > > > > > I don’t think this commit actually requires a vote.    
However,
> > > > > > it does seem like a significant change to the build process
> > > > > > that ought to be reviewed.     So I propose to treat this
as a
> > > > > > “lazy consensus” vote, and will commit the change to the
2.2
> > > > > > branch if there are no objections in 72 hours (i.e. 1730UTC
> > > > > > 20140105).
> > > > > > 
> > > > > > At the same time, based on discussions over on
> > > > > > general@incubator.apache.org, I’ll withdraw my assertion that
> > > > > > we can’t have jars in svn.     Those interested may want
to
> > > > > > check out the thread at
> > > > > > http://mail-archives.apache.org/mod_mbox/incubator-general/201312.mbox/%3C01B04CC4-95B8-4A39-BC16-04BAA4269B65%40stratuscom.com%3E
> > > > > > 
> > > > > > Cheers,
> > > > > > 
> > > > > > Greg.
> > > > > > 
> > > > > > On Jan 2, 2014, at 12:05 PM, Greg Trasuk (JIRA)
> > > > > > <jira@apache.org> wrote:
> > > > > > 
> > > > > > > 
> > > > > > > [
> > > > > > > https://issues.apache.org/jira/browse/RIVER-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > > > > > > ]
> > > > > > > 
> > > > > > > Greg Trasuk updated RIVER-432:
> > > > > > > ------------------------------
> > > > > > > 
> > > > > > > Attachment: river-2_2_remove_jars.diff
> > > > > > > 
> > > > > > > The attached patch for the 2.2 branch does the following:
> > > > > > > - removes the 'asm' directory and 'tests/lib' directories
> > > > > > > which currently contain the asm library, mockito, and junit
> > > > > > > jars. - Modifies 'build.xml', 'common.xml', and adds
> > > > > > > 'ivy.xml' so that the Apache Ivy ant plugin is downloaded
at
> > > > > > > build time, and then used to retrieve the libraries
> > > > > > > mentioned above from Maven Central.     This removes
the need
> > > > > > > to have the jar files in svn. - Removes (as per discussion
> > > > > > > http://mail-archives.apache.org/mod_mbox/river-dev/201211.mbox/%3C509B99E3.6080800%40qcg.nl%3E)
> > > > > > > the VelocityConfigBuilder, and associated Velocity jars. 
 
> > > > > > > Note that the 'extras' folder is not present in the 2.2
> > > > > > > branch, so Sim's last comments in the thread do not apply.
> > > > > > > 
> > > > > > > > Jar files in svn and src distributions
> > > > > > > > --------------------------------------
> > > > > > > > 
> > > > > > > > Key: RIVER-432
> > > > > > > > URL: https://issues.apache.org/jira/browse/RIVER-432
> > > > > > > > Project: River
> > > > > > > > Issue Type: Bug
> > > > > > > > Reporter: Greg Trasuk
> > > > > > > > Attachments: river-2_2_remove_jars.diff
> > > > > > > > 
> > > > > > > > 
> > > > > > > > Recent traffic on the incubator lists has pointed
out that
> > > > > > > > including jar files for dependencies in the subversion
> > > > > > > > repository and the source distributions is against
Apache
> > > > > > > > policy. In River, the following libraries appear in
the
> > > > > > > > Subversion repository and the source distributions
(these
> > > > > > > > are from trunk, a smaller set appear in the 2.2 branch):
> > > > > > > > animal-sniffer asm bouncy-castle dnsjava high-scale-lib
> > > > > > > > rc-libs
> > > > > > > > velocity
> > > > > > > > They all have to go.     What are we using them
for?     As I
> > > > > > > > understand it, we were going to remove the
> > > > > > > > VelocityConfigurationBuilder, so that's not a problem. 
 
> > > > > > > > Some of the others are available from Maven Central,
so we
> > > > > > > > can get them at build time using Ivy or another build
> > > > > > > > tool.     Which ones are actually required?   
 And where did
> > > > > > > > they come from?
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > --
> > > > > > > This message was sent by Atlassian JIRA
> > > > > > > (v6.1.5#6160)
> > > > > > 
> > > > > 
> > > > > 
> > > > > -- 
> > > > > QCG, Software voor het MKB, 071-5890970, http://www.qcg.nl
> > > > > Quality Consultancy Group b.v., Leiderdorp, Kvk Den Haag:
> > > > > 28088397
> > > > 
> > > 
> > 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message