edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dale LaBossiere <dml.apa...@gmail.com>
Subject Re: mvn, Edgent samples, and the broader app development and packaging topic
Date Mon, 26 Jun 2017 21:26:01 GMT
Chris, thanks for helping to work through this.

> On Jun 25, 2017, at 9:44 AM, Christofer Dutz <christofer.dutz@c-ware.de> wrote:
> 
> Hi Dale,
> 
> I just paused my work on the classpath generation and simply added the maven-shade-plugin
which creates uber-jars, which produces jars that contain not only the modules content, but
also includes all dependencies into one big fat jar … I also added a sample start script
to the topology project. 
> 
> Would this do as a startging point?
I pulled down the latest PR changes and built for all platforms.
I can’t seem to eliminate all errors in a Eclipse dev context (CLI builds are fine).  I’ll
follow up with that separately.

> 
> Regarding your points:
> First section:
> - Well it now produces an additional self-contained jar additionally
Observation: the uber-jar contains the extracted dependent component/jar contents as opposed
to containing the actual dependent jars.  Any benefits of one form over there other?
There’s some “xsl” stuff in the samples pom - part of your paused classpath generation
work that should be removed?

> - When using Maven you usually don’t use this model of having something installed,
it is possible using the “system” scope in dependencies and providing the systemPath element
pointing to some “${edgent.home}/my-edgent-lib-1.2.0.jar” but that wouldn’t be good
style. Usually the single-source for the artifacts would be the maven local repo on that device.
I’m confident I don’t know the various ways in which organizations will want to produce
edge devices that contain Edgent-based apps :-)  So I’m hesitant to constrain things / require
that maven technology be resident on the device.  That’s what’s motivates me to think
we need to ~easily support “install” based cases… and I’m not confident that supplying
“uber-jar” as the only initial alternative will be OK.

I guess if we also provide a way to construct a dir(s) containing all of the edgent jars dependencies
(not just a single sample’s dependencies yet) then maybe that and the uber-jar alternative
will be good enough to start.  At least the user won’t have to scratch their head on how
to get the jars out of the local repo if they really need them.

> Second section:
> - The shade plugin would work on all platforms. All I would have to do, is to add the
plugin configuration to the other platforms.
> - Having jars for each example would require splitting them up into separate maven modules.
I don’t think that’s a good idea. I’d prefer a start script that lets the user choose
the sample to run. I know this would require adjusting the start script whenever a sample
is added/deleted/changed, but it would reduce the number of these super-fat uber jars.
At one level I appreciate what you’re saying.  Maybe theres a need for another type of sample
that differs from this collection?  Let me try to clarify…

At a high level it feels like we need at least one sample project that we can say “copy
this sample project (source, pom, scripts if any) to jump-start the development of your Edgent-based
app”.  Do you think that what you’re considering will serve that purpose well?

Let’s fully treat samples like app code that a user would write.
Don't build them as part of edgent-parent, don’t depend at all on edgent-parent/pom.xml.
 Depending on a common sample-parent pom is OK.
Don’t include sample jars in a binary release.
Get things into a shape where samples could live in a separate repo.

I don’t think “create uber jar” has to be included in a sample default build config.
 It might be fine / preferable? for the user to have to specially request it.

The connector-sample is an example of why a multi-sample uber-jar isn’t a great.  If someone
wants to try the mqtt connector sample on a device where they didn’t build the sample, the
easiest thing would be to get the sample’s uber-jar to the device.  But that uber-jar is
large ~22MB.  I’m pretty sure that a lot of that is associated with other connector samples
(e.g., the kafka-connector’s dependent jars).  As a user of a sample I think I’d appreciate
being able to bring over a much smaller jar.  And being able to show a ~minimal uber-jar for
an app might start to help with questions about “size”.

> - If the user ran the maven build (mvn clean package) any user would be able to run the
samples.
> - IntelliJ was able to immediately run every example I tried, even before the shade plugin
was added and I think Eclipse this should also work.
Yup, I knew that Eclipse already supported that (and assumed similarly for INtelliJ).  Was
just trying to be complete :-)

> 
> Regarding the binary distribution. I think we should put non-edgent libs in a separate
directory. Mainly because this way we clearly draw a line to what’s our stuff and which
is from others (even if these “others” are also ASF projects)
OK, seems reasonable.

> 
> Chris
> 
> Am 23.06.17, 18:40 schrieb "Dale LaBossiere" <dml.apache@gmail.com>:
> 
>    Chris, thanks for the update.
> 
>    First, and most importantly, I hope you have a great weekend! :-)
> 
>    At a high level, I was thinking there’s value in a couple of build/packaging approaches.
>    - create a totally self contained executable-jar
>    - something that supports a model where a device may have a separately installed and
managed Edgent that any/all apps on the device use.
>    - a twist on the former where just the Edgent jars needed by the app are copied to
a local dir and bundled up
> 
>    For the samples:
>    - we should provide tooling that supports all of the above
>    - the tooling should be able to generate j8, j7, or android
>    - it might be good if each sample had its own pom.xml / generated .jar rather than
one bit one
>    - the user should be able to easily run a CLI-built sample locally (no need to actually
copy things to a “device”)
>    - the user should be able to easily import a sample into a maven-integrated IDE and
build and run it
>    - no need to include/demo building without mvn tooling
> 
>    Expanding on that...
> 
>    - create a totally self contained executable-jar
>      This is what you get from Eclipse when selecting a maven-based Edgent app project
>      and doing Export… > Runnable JAR > “package required libraries into jar”.
>      The user just gets that jar onto their device and a “java -jar <the-jar>”
and starts their app.  
>      Very simple.  No additional work to get/install Edgent.  
>      Only the app’s dependent Edgent jars (and deps) are included in the jar - i.e.,
it’s ~minimal/
>      This is perhaps an actual production mode of use for simple "single Edgent app"
devices.
> 
>    - something that supports a model where a device may have a separately installed and
managed Edgent that any/all apps on the device use.
>      This has some characteristics that you mentioned. 
>      I’m not sure how important it is to separate the Edgent vs “ext” jars/directories.
>      Unless there’s a compelling reason to do so, and it’s really not that much effort/complication,
I think I’d just use a single dir for all :-)
> 
>      Provide some cmd that populates a local dir with *all* the Edgent jars and their
deps (the analog of a full Edgent binary bundle).
>      Provide an edgent-classpath.sh cmd that returns a CLASSPATH value that simply includes
everything in that dir
>      Bundle up that that local dir and the edgent-classpath script.
>      The user needs to get bundle onto the device and unpack it.
>      Note, building the app wouldn’t use this dir/bundle, it’s just for deployment.
>     (well, a user that had non-maven-intergrated app built tooling could use the generated
dir)
> 
>      Create a jar containing just the Edgent app; make its manifest have and entry for
main class.
>      Provide a run-app.sh script that essentially just does:
>    	export CLASSPATH=`edgent-classpath.sh` # user can always augment this stmt if they
have other things to add
>            java -jar <the-edgent-app-jar>   # start the app
>      Bundle up that jar and the run script.
>      The user needs to get bundle onto the device and unpack it.
>      The app is started via “./run-app.sh”
> 
>    - a twist on the former where just the Edgent jars needed by the app are copied to
a local dir and bundled up
>      This leverages mvn to create the ~minimal set of Edgent jars and their deps needed
by the app.
>      The dir contains what “mvn dependency:copy-dependency” creates.
>      This seems to be the same thing as created by Eclipse   <project> > Export
… > Runnable JAR > “copy required jars into sub-folder”.
>      (Note, unlike that Eclipse export, the app’s jar would NOT contain a manifest
classpath.  The edgent-classpath.sh scheme would be used.)
>      The dir can be bundled up and brought to the device and unpacked.
> 
>    Hope that’s all mostly making sense / sensible :-)
> 
>    — Dale
> 
>> On Jun 23, 2017, at 3:39 AM, Christofer Dutz <christofer.dutz@c-ware.de> wrote:
>> ...Just a little on what I’m currently working on. 
>> The maven dependency plugin is able to create an xml representation of a projects
dependencies. I’m currently trying to create a build that pipes that through an XSLT and
hereby creates a start-script for the example they are built in. This will refer to a directory
structure in which all Edgent artifacts are output into one directory and all non-edgent dependencies
to an “ext” subdirectory within that lib directory. The scripts should be able to correctly
build an applications classpath based on that.
> 
> 
> 


Mime
View raw message