manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: MCF 2.x binary package directory structure
Date Fri, 26 Sep 2014 09:16:32 GMT
CONNECTORS-1048 created for including jar versions.
Karl

On Fri, Sep 26, 2014 at 5:03 AM, Karl Wright <daddywri@gmail.com> wrote:

> Hi Abe-san,
>
> It seems to me that we are discussing some distinct and independent
> improvements, so maybe I will open specific tickets for each one and we can
> work on things one at a time.  But, first:
>
> bq. I just simplified directories since other Apache projects look like
> much simpler.
>
> That's true, but I don't know of any other Apache project which attempts
> to handle conditional compilation to the degree that we have to in
> ManifoldCF.  So I think we should expect things to be more complicated. :-)
>
> bq. BTW, for a few jars(oracle,mysql,alfresco,jcifs,livelink,jtds and
> other),  to create *-proprietary directories is awkward, isn't  it?
>
> I've never particularly liked the -proprietary paradigm, but I could think
> of no other way.  The reasons behind it was that we *must* legally exclude
> anything proprietary from the distribution.  By making sure that every
> proprietary jar is in a well-labeled directory, I can simply make sure all
> such directories are excluded.  But that also includes wars.  Since wars
> must include all jars, that meant that the build had to create two
> different sets of wars, and put them  in different directories.  The only
> other alternative is to guarantee that we always build our release
> candidates from a fresh checkout each time, and that seemed risky.
>
> I agree that it would be more convenient for the user if our build created
> one set of wars based on what was included by the user, and that we shipped
> just the minimal binary that didn't include proprietary code.  If we can
> solve the war problem, we can do that.
>
> Karl
>
>
>
> On Fri, Sep 26, 2014 at 12:51 AM, Shinichiro Abe <
> shinichiro.abe.1@gmail.com> wrote:
>
>> Hi Karl,
>>
>> Thanks for the reply.
>>
>> > (1) I don't understand what you mean by, "because build.xml in
>> framework is
>> > currently too long to include class
>> > paths in lib".  Can you clarify?
>>
>> Rerated to (5) , manifest-cp configuration is not cool at all.
>>
>> https://svn.apache.org/viewvc/manifoldcf/trunk/framework/build.xml?view=markup#l1343
>> If new jars introduced with new feature, manifest-cp-proprietary must be
>> tweaked, too.
>> (i18n properties files also bother us.)
>>
>> Alternatively, the ant task below is useful.
>>
>> common-build.xml:  Add artifact-version to dest.
>> + <get
>> src="${maven-base-url}/${project-path}/${artifact-name}/${artifact-version}/${artifact-name}-${artifact-version}.${artifact-type}"
>> dest="${target}/${artifact-name}-${artifact-version}.${artifact-type}"/>
>>
>> framework/build.xml: New manifest-cp property setting works fine.
>>
>> -          <property name="manifest-cp-74" value="${manifest-cp-73}
>> ../lib/xmlsec.jar"/>
>> -           <property name="manifest-cp-75" value="${manifest-cp-74}
>> ../lib/opensaml.jar"/>
>>
>> +        <property name="liblocation" location="../lib" />
>> +        <path id="lib-jars">
>> +            <fileset dir="${liblocation}" includes="*.jar"/>
>> +        </path>
>> +        <pathconvert property="manifest-cp" refid="lib-jars"
>> targetos="unix" pathsep=" ">
>> +            <map from="${liblocation}" to="../lib"/>
>> +            <map from="\" to="/"/>
>> +        </pathconvert>
>>
>> +        <!-- <property name="manifest-cp" value="${manifest-cp-75}"/> -->
>>
>> > (2) Some of these suggestions seem to be making distinctions between
>> files
>> > and directories that I don't understand the reason for.
>>
>> 1. I came to mind CONNECTORS-345.
>> 2. Currently I have to create scripts in production:
>>  startup.sh --
>>     java -jar start.jar
>>     echo $! > mcf.pid
>>  shutdown.sh --
>>     kill cat(mcf.pid) <-- MCF's embedded Jetty example can't call
>> agent-shutdown AFAIK but this script isn't cool.
>>
>> 3. start.jar command line options are useful.
>> http://www.eclipse.org/jetty/documentation/current/start-jar.html#d0e8418
>>
>> 4. If we can separate Jetty-lib and mcf-lib, we could deploy Tomcat. Of
>> course combined.war can do that though.
>> I thought additional jars could copy to another directory, and might load
>> these using Custom ClassLoader.
>> http://www.eclipse.org/jetty/documentation/current/jetty-classloading.html
>>
>> 5. But If users don't want to config jetty.xml(e.g. thread pool,
>> requestHeaderSize and so on in jetty.xml --
>> http://wiki.eclipse.org/Jetty/Howto/Configure_Connectors),
>> I will drop these suggestions.
>>
>> > (3) I agree with "connector-specific-processes" and "plugins".  But
>> other
>> > hierarchy additions seem less helpful, such as hiding the examples under
>> > additional levels of hierarchy.  I think it should be possible
>> immediately
>> > for a user to see what examples are available.  But maybe we could
>> change
>> > the names to be clearer.
>> Yes. Especially I want to change name *-proprietary into something like
>> *-extra because "proprietary".length() is long.
>>
>> BTW, for a few jars(oracle,mysql,alfresco,jcifs,livelink,jtds and
>> other),  to create *-proprietary directories is awkward, isn't  it?
>> I know CONNECTORS-402 intent, but more simplified, can we say that "if
>> you use these jars,
>> please get source and run ''ant make-deps build"? -- I usually say so, no
>> problems occur for now.
>>
>> > (4) Rather than group together xxx and xxx-proprietary, and yyy and
>> > yyy-proprietary, it would be more appropriate to have a "proprietary"
>> > directory and an "open" directory", with xxx and yyy under them.
>>
>> Ok.
>> But I want to know, can we deprecate example and  move
>> example-proprietary to example?
>> I'd like to merge these examples. If proprietary jars exists, MCF just
>> have to load from extra-dir.
>> I assume that we can have extra-lib which is optional, If it exists, MCF
>> will load any Jars
>> found in this directory and use them to resolve connectors specified in
>> connectors.xml.
>> (Solr has plugin class loader at $SOLR_HOME/lib)
>>
>> I just simplified directories since other Apache projects look like much
>> simpler.
>>
>> Regards,
>> Shinichiro Abe.
>>
>> On 2014/09/26, at 3:01, Karl Wright <daddywri@gmail.com> wrote:
>>
>> > Hi Abe-san,
>> >
>> > Some comments:
>> >
>> > (1) I don't understand what you mean by, "because build.xml in
>> framework is
>> > currently too long to include class
>> > paths in lib".  Can you clarify?
>> >
>> > (2) Some of these suggestions seem to be making distinctions between
>> files
>> > and directories that I don't understand the reason for.  For example:
>> "in
>> > process-single directory we can
>> > create lib dir(for jetty jars), lib/ext dir(for logger jars), resource
>> > dir(for logging.ini) and etc dir(for jetty.xml)."  Why separate jars in
>> > this way?  They are all necessary at the root level to run the
>> example.  I
>> > would not understand where to add a new jar with this arrangement
>> because
>> > the meaning of the directories is unclear.
>> >
>> > (3) I agree with "connector-specific-processes" and "plugins".  But
>> other
>> > hierarchy additions seem less helpful, such as hiding the examples under
>> > additional levels of hierarchy.  I think it should be possible
>> immediately
>> > for a user to see what examples are available.  But maybe we could
>> change
>> > the names to be clearer.
>> >
>> > (4) Rather than group together xxx and xxx-proprietary, and yyy and
>> > yyy-proprietary, it would be more appropriate to have a "proprietary"
>> > directory and an "open" directory", with xxx and yyy under them.
>> >
>> > (5) Putting version numbers on jars is difficult in some cases,
>> especially
>> > in construction of start.jar, because the ant methods for constructing
>> > start.jar are poor.  The version of each jar would need to be defined
>> > globally in the ant build, and included whenever the jar is referenced.
>> >
>> > Karl
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message