manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: MCF 2.x binary package directory structure
Date Fri, 26 Sep 2014 09:28:34 GMT
I've replied to the jetty vs. tomcat issue in the CONNECTORS-345 comment
thread.
Thanks!
Karl

On Fri, Sep 26, 2014 at 5:22 AM, Karl Wright <daddywri@gmail.com> wrote:

> I've moved CONNECTORS-345 to "Fix in 2.0" and attached your comments.
> Let's talk about the jetty deployment issues in that ticket thread.
>
> Karl
>
>
> On Fri, Sep 26, 2014 at 5:16 AM, Karl Wright <daddywri@gmail.com> wrote:
>
>> CONNECTORS-1048 created for including jar versions.
>> Karl
>>
>> On Fri, Sep 26, 2014 at 5:03 AM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>> Hi Abe-san,
>>>
>>> It seems to me that we are discussing some distinct and independent
>>> improvements, so maybe I will open specific tickets for each one and we can
>>> work on things one at a time.  But, first:
>>>
>>> bq. I just simplified directories since other Apache projects look like
>>> much simpler.
>>>
>>> That's true, but I don't know of any other Apache project which attempts
>>> to handle conditional compilation to the degree that we have to in
>>> ManifoldCF.  So I think we should expect things to be more complicated. :-)
>>>
>>> bq. BTW, for a few jars(oracle,mysql,alfresco,jcifs,livelink,jtds and
>>> other),  to create *-proprietary directories is awkward, isn't  it?
>>>
>>> I've never particularly liked the -proprietary paradigm, but I could
>>> think of no other way.  The reasons behind it was that we *must* legally
>>> exclude anything proprietary from the distribution.  By making sure that
>>> every proprietary jar is in a well-labeled directory, I can simply make
>>> sure all such directories are excluded.  But that also includes wars.
>>> Since wars must include all jars, that meant that the build had to create
>>> two different sets of wars, and put them  in different directories.  The
>>> only other alternative is to guarantee that we always build our release
>>> candidates from a fresh checkout each time, and that seemed risky.
>>>
>>> I agree that it would be more convenient for the user if our build
>>> created one set of wars based on what was included by the user, and that we
>>> shipped just the minimal binary that didn't include proprietary code.  If
>>> we can solve the war problem, we can do that.
>>>
>>> Karl
>>>
>>>
>>>
>>> On Fri, Sep 26, 2014 at 12:51 AM, Shinichiro Abe <
>>> shinichiro.abe.1@gmail.com> wrote:
>>>
>>>> Hi Karl,
>>>>
>>>> Thanks for the reply.
>>>>
>>>> > (1) I don't understand what you mean by, "because build.xml in
>>>> framework is
>>>> > currently too long to include class
>>>> > paths in lib".  Can you clarify?
>>>>
>>>> Rerated to (5) , manifest-cp configuration is not cool at all.
>>>>
>>>> https://svn.apache.org/viewvc/manifoldcf/trunk/framework/build.xml?view=markup#l1343
>>>> If new jars introduced with new feature, manifest-cp-proprietary must
>>>> be tweaked, too.
>>>> (i18n properties files also bother us.)
>>>>
>>>> Alternatively, the ant task below is useful.
>>>>
>>>> common-build.xml:  Add artifact-version to dest.
>>>> + <get
>>>> src="${maven-base-url}/${project-path}/${artifact-name}/${artifact-version}/${artifact-name}-${artifact-version}.${artifact-type}"
>>>> dest="${target}/${artifact-name}-${artifact-version}.${artifact-type}"/>
>>>>
>>>> framework/build.xml: New manifest-cp property setting works fine.
>>>>
>>>> -          <property name="manifest-cp-74" value="${manifest-cp-73}
>>>> ../lib/xmlsec.jar"/>
>>>> -           <property name="manifest-cp-75" value="${manifest-cp-74}
>>>> ../lib/opensaml.jar"/>
>>>>
>>>> +        <property name="liblocation" location="../lib" />
>>>> +        <path id="lib-jars">
>>>> +            <fileset dir="${liblocation}" includes="*.jar"/>
>>>> +        </path>
>>>> +        <pathconvert property="manifest-cp" refid="lib-jars"
>>>> targetos="unix" pathsep=" ">
>>>> +            <map from="${liblocation}" to="../lib"/>
>>>> +            <map from="\" to="/"/>
>>>> +        </pathconvert>
>>>>
>>>> +        <!-- <property name="manifest-cp" value="${manifest-cp-75}"/>
>>>> -->
>>>>
>>>> > (2) Some of these suggestions seem to be making distinctions between
>>>> files
>>>> > and directories that I don't understand the reason for.
>>>>
>>>> 1. I came to mind CONNECTORS-345.
>>>> 2. Currently I have to create scripts in production:
>>>>  startup.sh --
>>>>     java -jar start.jar
>>>>     echo $! > mcf.pid
>>>>  shutdown.sh --
>>>>     kill cat(mcf.pid) <-- MCF's embedded Jetty example can't call
>>>> agent-shutdown AFAIK but this script isn't cool.
>>>>
>>>> 3. start.jar command line options are useful.
>>>>
>>>> http://www.eclipse.org/jetty/documentation/current/start-jar.html#d0e8418
>>>>
>>>> 4. If we can separate Jetty-lib and mcf-lib, we could deploy Tomcat. Of
>>>> course combined.war can do that though.
>>>> I thought additional jars could copy to another directory, and might
>>>> load these using Custom ClassLoader.
>>>>
>>>> http://www.eclipse.org/jetty/documentation/current/jetty-classloading.html
>>>>
>>>> 5. But If users don't want to config jetty.xml(e.g. thread pool,
>>>> requestHeaderSize and so on in jetty.xml --
>>>> http://wiki.eclipse.org/Jetty/Howto/Configure_Connectors),
>>>> I will drop these suggestions.
>>>>
>>>> > (3) I agree with "connector-specific-processes" and "plugins".  But
>>>> other
>>>> > hierarchy additions seem less helpful, such as hiding the examples
>>>> under
>>>> > additional levels of hierarchy.  I think it should be possible
>>>> immediately
>>>> > for a user to see what examples are available.  But maybe we could
>>>> change
>>>> > the names to be clearer.
>>>> Yes. Especially I want to change name *-proprietary into something like
>>>> *-extra because "proprietary".length() is long.
>>>>
>>>> BTW, for a few jars(oracle,mysql,alfresco,jcifs,livelink,jtds and
>>>> other),  to create *-proprietary directories is awkward, isn't  it?
>>>> I know CONNECTORS-402 intent, but more simplified, can we say that "if
>>>> you use these jars,
>>>> please get source and run ''ant make-deps build"? -- I usually say so,
>>>> no problems occur for now.
>>>>
>>>> > (4) Rather than group together xxx and xxx-proprietary, and yyy and
>>>> > yyy-proprietary, it would be more appropriate to have a "proprietary"
>>>> > directory and an "open" directory", with xxx and yyy under them.
>>>>
>>>> Ok.
>>>> But I want to know, can we deprecate example and  move
>>>> example-proprietary to example?
>>>> I'd like to merge these examples. If proprietary jars exists, MCF just
>>>> have to load from extra-dir.
>>>> I assume that we can have extra-lib which is optional, If it exists,
>>>> MCF will load any Jars
>>>> found in this directory and use them to resolve connectors specified in
>>>> connectors.xml.
>>>> (Solr has plugin class loader at $SOLR_HOME/lib)
>>>>
>>>> I just simplified directories since other Apache projects look like
>>>> much simpler.
>>>>
>>>> Regards,
>>>> Shinichiro Abe.
>>>>
>>>> On 2014/09/26, at 3:01, Karl Wright <daddywri@gmail.com> wrote:
>>>>
>>>> > Hi Abe-san,
>>>> >
>>>> > Some comments:
>>>> >
>>>> > (1) I don't understand what you mean by, "because build.xml in
>>>> framework is
>>>> > currently too long to include class
>>>> > paths in lib".  Can you clarify?
>>>> >
>>>> > (2) Some of these suggestions seem to be making distinctions between
>>>> files
>>>> > and directories that I don't understand the reason for.  For example:
>>>> "in
>>>> > process-single directory we can
>>>> > create lib dir(for jetty jars), lib/ext dir(for logger jars), resource
>>>> > dir(for logging.ini) and etc dir(for jetty.xml)."  Why separate jars
>>>> in
>>>> > this way?  They are all necessary at the root level to run the
>>>> example.  I
>>>> > would not understand where to add a new jar with this arrangement
>>>> because
>>>> > the meaning of the directories is unclear.
>>>> >
>>>> > (3) I agree with "connector-specific-processes" and "plugins".  But
>>>> other
>>>> > hierarchy additions seem less helpful, such as hiding the examples
>>>> under
>>>> > additional levels of hierarchy.  I think it should be possible
>>>> immediately
>>>> > for a user to see what examples are available.  But maybe we could
>>>> change
>>>> > the names to be clearer.
>>>> >
>>>> > (4) Rather than group together xxx and xxx-proprietary, and yyy and
>>>> > yyy-proprietary, it would be more appropriate to have a "proprietary"
>>>> > directory and an "open" directory", with xxx and yyy under them.
>>>> >
>>>> > (5) Putting version numbers on jars is difficult in some cases,
>>>> especially
>>>> > in construction of start.jar, because the ant methods for constructing
>>>> > start.jar are poor.  The version of each jar would need to be defined
>>>> > globally in the ant build, and included whenever the jar is
>>>> referenced.
>>>> >
>>>> > Karl
>>>>
>>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message