manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: MCF 2.x binary package directory structure
Date Fri, 26 Sep 2014 09:03:19 GMT
Hi Abe-san,

It seems to me that we are discussing some distinct and independent
improvements, so maybe I will open specific tickets for each one and we can
work on things one at a time.  But, first:

bq. I just simplified directories since other Apache projects look like
much simpler.

That's true, but I don't know of any other Apache project which attempts to
handle conditional compilation to the degree that we have to in
ManifoldCF.  So I think we should expect things to be more complicated. :-)

bq. BTW, for a few jars(oracle,mysql,alfresco,jcifs,livelink,jtds and
other),  to create *-proprietary directories is awkward, isn't  it?

I've never particularly liked the -proprietary paradigm, but I could think
of no other way.  The reasons behind it was that we *must* legally exclude
anything proprietary from the distribution.  By making sure that every
proprietary jar is in a well-labeled directory, I can simply make sure all
such directories are excluded.  But that also includes wars.  Since wars
must include all jars, that meant that the build had to create two
different sets of wars, and put them  in different directories.  The only
other alternative is to guarantee that we always build our release
candidates from a fresh checkout each time, and that seemed risky.

I agree that it would be more convenient for the user if our build created
one set of wars based on what was included by the user, and that we shipped
just the minimal binary that didn't include proprietary code.  If we can
solve the war problem, we can do that.

Karl



On Fri, Sep 26, 2014 at 12:51 AM, Shinichiro Abe <shinichiro.abe.1@gmail.com
> wrote:

> Hi Karl,
>
> Thanks for the reply.
>
> > (1) I don't understand what you mean by, "because build.xml in framework
> is
> > currently too long to include class
> > paths in lib".  Can you clarify?
>
> Rerated to (5) , manifest-cp configuration is not cool at all.
>
> https://svn.apache.org/viewvc/manifoldcf/trunk/framework/build.xml?view=markup#l1343
> If new jars introduced with new feature, manifest-cp-proprietary must be
> tweaked, too.
> (i18n properties files also bother us.)
>
> Alternatively, the ant task below is useful.
>
> common-build.xml:  Add artifact-version to dest.
> + <get
> src="${maven-base-url}/${project-path}/${artifact-name}/${artifact-version}/${artifact-name}-${artifact-version}.${artifact-type}"
> dest="${target}/${artifact-name}-${artifact-version}.${artifact-type}"/>
>
> framework/build.xml: New manifest-cp property setting works fine.
>
> -          <property name="manifest-cp-74" value="${manifest-cp-73}
> ../lib/xmlsec.jar"/>
> -           <property name="manifest-cp-75" value="${manifest-cp-74}
> ../lib/opensaml.jar"/>
>
> +        <property name="liblocation" location="../lib" />
> +        <path id="lib-jars">
> +            <fileset dir="${liblocation}" includes="*.jar"/>
> +        </path>
> +        <pathconvert property="manifest-cp" refid="lib-jars"
> targetos="unix" pathsep=" ">
> +            <map from="${liblocation}" to="../lib"/>
> +            <map from="\" to="/"/>
> +        </pathconvert>
>
> +        <!-- <property name="manifest-cp" value="${manifest-cp-75}"/> -->
>
> > (2) Some of these suggestions seem to be making distinctions between
> files
> > and directories that I don't understand the reason for.
>
> 1. I came to mind CONNECTORS-345.
> 2. Currently I have to create scripts in production:
>  startup.sh --
>     java -jar start.jar
>     echo $! > mcf.pid
>  shutdown.sh --
>     kill cat(mcf.pid) <-- MCF's embedded Jetty example can't call
> agent-shutdown AFAIK but this script isn't cool.
>
> 3. start.jar command line options are useful.
> http://www.eclipse.org/jetty/documentation/current/start-jar.html#d0e8418
>
> 4. If we can separate Jetty-lib and mcf-lib, we could deploy Tomcat. Of
> course combined.war can do that though.
> I thought additional jars could copy to another directory, and might load
> these using Custom ClassLoader.
> http://www.eclipse.org/jetty/documentation/current/jetty-classloading.html
>
> 5. But If users don't want to config jetty.xml(e.g. thread pool,
> requestHeaderSize and so on in jetty.xml --
> http://wiki.eclipse.org/Jetty/Howto/Configure_Connectors),
> I will drop these suggestions.
>
> > (3) I agree with "connector-specific-processes" and "plugins".  But other
> > hierarchy additions seem less helpful, such as hiding the examples under
> > additional levels of hierarchy.  I think it should be possible
> immediately
> > for a user to see what examples are available.  But maybe we could change
> > the names to be clearer.
> Yes. Especially I want to change name *-proprietary into something like
> *-extra because "proprietary".length() is long.
>
> BTW, for a few jars(oracle,mysql,alfresco,jcifs,livelink,jtds and other),
> to create *-proprietary directories is awkward, isn't  it?
> I know CONNECTORS-402 intent, but more simplified, can we say that "if you
> use these jars,
> please get source and run ''ant make-deps build"? -- I usually say so, no
> problems occur for now.
>
> > (4) Rather than group together xxx and xxx-proprietary, and yyy and
> > yyy-proprietary, it would be more appropriate to have a "proprietary"
> > directory and an "open" directory", with xxx and yyy under them.
>
> Ok.
> But I want to know, can we deprecate example and  move example-proprietary
> to example?
> I'd like to merge these examples. If proprietary jars exists, MCF just
> have to load from extra-dir.
> I assume that we can have extra-lib which is optional, If it exists, MCF
> will load any Jars
> found in this directory and use them to resolve connectors specified in
> connectors.xml.
> (Solr has plugin class loader at $SOLR_HOME/lib)
>
> I just simplified directories since other Apache projects look like much
> simpler.
>
> Regards,
> Shinichiro Abe.
>
> On 2014/09/26, at 3:01, Karl Wright <daddywri@gmail.com> wrote:
>
> > Hi Abe-san,
> >
> > Some comments:
> >
> > (1) I don't understand what you mean by, "because build.xml in framework
> is
> > currently too long to include class
> > paths in lib".  Can you clarify?
> >
> > (2) Some of these suggestions seem to be making distinctions between
> files
> > and directories that I don't understand the reason for.  For example: "in
> > process-single directory we can
> > create lib dir(for jetty jars), lib/ext dir(for logger jars), resource
> > dir(for logging.ini) and etc dir(for jetty.xml)."  Why separate jars in
> > this way?  They are all necessary at the root level to run the example.
> I
> > would not understand where to add a new jar with this arrangement because
> > the meaning of the directories is unclear.
> >
> > (3) I agree with "connector-specific-processes" and "plugins".  But other
> > hierarchy additions seem less helpful, such as hiding the examples under
> > additional levels of hierarchy.  I think it should be possible
> immediately
> > for a user to see what examples are available.  But maybe we could change
> > the names to be clearer.
> >
> > (4) Rather than group together xxx and xxx-proprietary, and yyy and
> > yyy-proprietary, it would be more appropriate to have a "proprietary"
> > directory and an "open" directory", with xxx and yyy under them.
> >
> > (5) Putting version numbers on jars is difficult in some cases,
> especially
> > in construction of start.jar, because the ant methods for constructing
> > start.jar are poor.  The version of each jar would need to be defined
> > globally in the ant build, and included whenever the jar is referenced.
> >
> > Karl
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message