edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christofer Dutz <christofer.d...@c-ware.de>
Subject Re: Understanding the snapshot and release process
Date Sun, 28 May 2017 15:12:06 GMT
The Maven build should work on the ASF Jenkins and hereby publish the snapshots. I think that
in the flex project I added this to ensure its added to the generated "site".

Which one is the hosted Jenkins you are referring to? The asf Jenkins?

And what should be the problem with Travis? Why are you using two ci servers?

Chris



Von meinem Samsung Galaxy Smartphone gesendet.


-------- Ursprüngliche Nachricht --------
Von: "John D. Ament" <johndament@apache.org>
Datum: 28.05.17 16:20 (GMT+01:00)
An: dev@edgent.apache.org
Betreff: Re: Understanding the snapshot and release process

The maven based build seems to be much more maintainable.  I'm glad you
guys came together to solve this problem :-)

RE distribution of snapshots.  We only presently support publishing via our
hosted jenkins, however Edgent currently builds in Travis.  So that would
need to be changed, or some other solution.

You also can drop the repositories/pluginRepositories sections of the new
pom.  Its not needed.


John

On Sun, May 28, 2017 at 10:03 AM Christofer Dutz <christofer.dutz@c-ware.de>
wrote:

> Hi guys,
>
> Strange people these Germans … if it rains, they complain about it and how
> much they would like to go swimming in the sun … now that it’s 32° and the
> sun is shining, no one wants to go swimming, cause it’s too fu***g hot ;-)
> … well this way I had some time to finish the maven migration … I just
> pushed the changes to the “feature/maven” branch of my fork on github:
> https://github.com/chrisdutz/incubator-edgent/tree/feature/maven
>
> Here a short list on what little monsters I found siting under some rocks:
>
> 1) Usage of test-jars --> Bad practice → Move code to separate test-util
> jar module
> 2) AppServiceTest in provides/direct relies on system-properties to load
> jar produced by another module → Ideally this should be refactored to find
> the jar on the classpath
> 3) HttpServerTest in console/server relies on console.war
> 4) console/servlets module compiled to something with a name completely
> unrelated to the project
> 5) test/svt requires samples project → Some modules rely on samples. While
> I think, this is ok for Samples themselves, nothing outside the samples
> should rely on samples (my opinion)
> 6) Almost all tests relying on successful SSL handshakes in
> wsclient-javax.websocket are failing (I’ll investigate this as soon as I
> have time, I could imagine this to be related to non US JVM encryption
> issues we are also seeing in the Flex project)
> 7) Some unit-tests should more be integration-tests
>
> I didn’t migrate the stuff in “platform” as I didn’t quite understand what
> they are used for and what they should do.
>
> Hope you like the changes. With a build like this it should be easy to
> setup the CI to auto distribute SNAPSHOTs and execute releases on ASF
> machines with a simple one or two-liner.
>
> Chris
>
>
> Am 27.05.17, 19:16 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:
>
>     I did have some free time this weekend, so I thought I’d give the
> Maven thing a try. And right now I haven finished, but I think I’m about
> 80% done. So far there have been one or two little monsters under some
> rocks … but they weren’t too nasty ;-)
>
>     Regarding the classpath stuff in the manifest: I really dislike this
> way of setting the classpath as it usually causes a lot of problems when
> using it. It requires libs to be in a predefined directory structure and
> follow a given naming convention. Try to use this in a project built with
> Maven (just picking this as I saw this would be a problem for a Maven built
> project)
>
>     Regarding tests: I did see that there are several Unit tests I would
> probably more call integration-tests. Especially the ones that require
> loading of a war (console.war). So I guess you have both some classical
> unit-tests as well as integration-tests … no need to treat them the same
> way. Maven ususally has the unit-test and the integration-test phase for
> exactly this.
>
>     Regarding jacoco: So I’ll activate that in the maven build too
>
>     Regarding automatic build: For Flex I setup the build to do a
> multibranch pipeline build. So whenever someone checks something in
> “develop”, “releae/*” or “feature/*” it automatically builds that
> particular branch. However develop and “release/*” are the only ones
> uploading SNAPSHOT versions to the ASF Nexus.
>
>     Regarding my commitment: I think Edgent is the tool I was looking for
> in order to make my SCADA project feature-perfect. So I have a strong
> interest in the project maturing. I do have plans for a longer involvement
> in the project. Probably I would be trying to create an open-source “plc”
> connector (which would probably have to be developed outside of Apache due
> to libraries still being GPL). But I do plan on staying around some time ;-)
>
>     Regarding the usage: Maven or Gradle based application developers
> could directly utilize SNAPSHOT and release artifacts as they are released.
> Additionally, I would create a so-called assembly that produces the same
> zip or tar-gz archives the users have been using. So there shouldn’t be too
> much of a change.
>
>     So now I’ll try to finish the rest of the connector modules maven
> conversion (
>
>     Chris
>
>     Am 25.05.17, 19:37 schrieb "Dale LaBossiere" <dml.apache@gmail.com>:
>
>         Awaiting others to chime in on this “switch to mvn” subject.
>         A “don’t care” (+/- 0) response is preferred over silence.
>
>         In the mean time...
>
>         > On May 24, 2017, at 11:41 AM, Christofer Dutz <
> christofer.dutz@c-ware.de> wrote:
>         > ...
>         > I just checked out everything and managed to get things imported
> in IntelliJ after a little struggle … this is a good job you did :-)
>         Can’t take all the credit (or blame :-)
>         >
>         > Right now, it seems as if there were Ant+Gradle+Eclipse build
> files in there. While it might seem ...
>         gradle is the only CLI way to build edgent.  Any ant build.xml
> files are either (a) residual cruft that should be removed (my bad) or (b)
> needed to leverage the ant machinery for invoking retrolambda in building
> java7/android compatible versions of the jars (machinery that wasn’t
> converted to pure gradle due to time/effort/value at the time).  Attempting
> to use ant at the top level tells you it doesn’t work :-)  As for Eclipse
> .project/.classpath, yup those are live and haven’t been a maintenance
> issue.
>
>         fwiw, I just removed all build.xml except the top level one and
> those under platform and “gradle release” worked fine so I’ll create a PR
> to clean them up as a first step.
>
>         > ...JavaDoc is generated automatically when running a release
> build together with the usual Maven project reports (is even configured in
> the apache parent POM together with rat, deployment etc.)
>         Javadoc was complicated by creating groupings as well as excluding
> it for non-API classes.
>
>         > ...I didn’t quite understand the “Manifest” thing, but the jar
> plugin does generate this with reasonable defaults, and can be extended to
> also export the dependencies into that (even if I don’t recommend that).
>         An edgent jar’s manifest class-path includes references to its
> immediate dependent edgent jars (not transitive) as well as references to
> its external jar dependencies (transitively).  I agree that "compiled in"
> references to specific versions of external dependencies may not be a great
> idea / is perhaps best eliminated.
>
>         My recollection is that by default gradle did not generate any
> manifest class-path.
>
>         e.g., just to make this a bit more concrete, this is from
> edgent.connectors.kafka.jar/MANIFEST.MF (see connectors/kafka/build.gradle
> for more info)
>
>         Class-Path: ../../../lib/edgent.api.topology.jar
> ../../../ext/gson-2.2
>          .4.jar ../../../ext/slf4j-api-1.7.12.jar
> ../../../ext/metrics-core-3.
>          1.2.jar ../ext/kafka_2.10-0.8.2.2.jar
> ../ext/kafka-clients-0.8.2.2.ja
>          r ../ext/log4j-1.2.16.jar ../ext/metrics-core-2.2.0.jar
> ../ext/scala-
>          library-2.10.4.jar ../ext/zkclient-0.3.jar
> ../ext/zookeeper-3.4.6.jar
>
>         edgent.api.topology.jar itself has lots of edgent jar dependencies
> (captured in its manifest) but that’s of no concern to the kafka
> connector.  I’m not sure maintaining this is a requirement and, at least
> for gradle, eliminating it would have greatly simplified things.
>
>         Test execution environment was also an issue.  perhaps more a
> result of structural issues.  i.e., the gradle config treated a component’s
> tests as unit tests, run against the associated component’s .class files.
> The ant system treated them more like integration tests, run against the
> jar files that would be bundled into a binary-release bundle; more like the
> environment that Edgent-based applications would use.  That was very
> important.  And due to some manual setup requirements for some connector
> tests are excluded from execution by default.
>
>         > …
>         > In the Flex project, I also setup the build to run SonarQube
> analysis and automatically generate the documentation from markdown and/or
> asciidoctor (which I think is very convenient) even automatically update
> and deploy the project website.
>
>         We include use of jacoco
>         We have travis integration for auto-PR validation.
>         It would be nice to have some periodic build/regression testing
> run on the main (master) branch.
>
>         > I could offer to create a fork on GitHub, create a feature
> branch there and try to whip up a set of poms that add Maven as fourth
> build system to the list … you could check it out and play around with it.
> But I’d only do this, if there is any interest in it.
>
>         I don’t blame you!
>
>         One worry I have is that it ends up like the gradle effort… it was
> ~trivial to get started but there was then a lot of effort required
> identify and flesh out equivalence with the ant based build result
> artifacts.
>         If you’re on board for the long haul in doing a conversion then
> that lessens my concern.
>
>         I’m on board to help but don’t want to inherit a “now finish it”
> task :-)
>
>         A full switch to a ~simple mvn-based build system would be the
> goal IMO ― we’d want to toss gradle. The effort ultimately also entails
> updating a bunch of doc.  Not a killer, just not to be overlooked.
>
>         It’s unclear to me how the Edgent-based app developers will be
> affected by all of this.  OK, they build their Edgent-based app with some
> tool that can utilize a maven repo.  Then what?
>
>         Today the story is simple: extract the binary-tgz (or a subset of
> it) on the target (edge device), copy your app code jar/classes to the
> target, set the CLASSPATH and go.  (assume we also eliminate
> building/distributing a binary-tgz)
>
>         Can elaborate on that part of the story?
>
>         Thanks!
>         ― Dale
>
>
>
>
>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message