incubator-connectors-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <>
Subject Order of development?
Date Mon, 01 Feb 2010 13:15:13 GMT

Once the Apache IP committee is done with the LCF grant, please note I've entered a number
of tickets for work that would need 
to be done immediately after the software hits Apache's SVN.

It seems to me that the highest priority of these is getting some set of starting ant build
scripts created.  That way, 
committers will be in a position to not break the world too badly when we start de-MetaCarta-ifying
the code itself.  I think it 
would also be a good idea to settle on a directory structure at this time, so that we all
know where we are heading, no?

The current structure of the granted code is as follows:

- mcdoc contains documentation
- mcsqa contains example tests
- upstream-diffs contains instructions for how to include or modify upstream packages
- products/connectors contains the framework itself, and all individual connectors, each with
their own subdirectory underneath
- products/libapache-mod-authz-annotate contains the code for the Apache2 mod-aa module that
does the search-side security piece
- products/postgres-config contains a package that should be an example of how to set up postgresql
properly (on debian systems, 
- products/java-environment contains a package that manages class paths for all the moving
parts, on debian systems

Inside the products/connectors/framework directory, there are three overall sections: a "core
functionality" section (in 
java-common), which contains both UI common functionality as well as crawler common functionality;
an "agent framework" section, 
in java-agents, which defines the concept of output connectors, and provides functionality
concerned with handling document 
ingestion and removal from an output connection; and a "crawler agent" section, under "crawler",
which does everything else. 
The UI component is lumped together with the "crawler agent" section right now, but probably
is logically separable.  The 
authority service and web application are also lumped in this bin, but should also be separated,

I'd like therefore to propose that the guts of the "framework" directory be rejiggered as

- Create a new "crawler-ui" directory, consisting of the stuff from java-common/ui, and the
crawler UI jsp code
- Create a new "authority-webapp" directory, consisting of the authority service web application
servlet code
- Move the framework/crawler/pullagent directory to framework/pull-agent
- If there aren't any interdependencies (I don't think there are, but I can't be sure yet),
create a new "authority-service" 
directory, containing the stuff currently under framework/crawler/pullagent/com/metacarta/authorities.
 This will introduce a 
new .jar file, but I think that's appropriate.

There may also be some rejiggering of individual connector directory hierarchies, but I haven't
thought as much about that yet.

Thoughts?  Comments?


View raw message