incubator-connectors-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Lucene Connector Framework > How to Write a Repository Connector
Date Tue, 09 Mar 2010 14:02:00 GMT
Space: Lucene Connector Framework (http://cwiki.apache.org/confluence/display/CONNECTORS)
Page: How to Write a Repository Connector (http://cwiki.apache.org/confluence/display/CONNECTORS/How+to+Write+a+Repository+Connector)

Added by Karl Wright:
---------------------------------------------------------------------
h1. Writing a Repository Connector

A repository connector furnishes the mechanism for obtaining documents, metadata, and authority
tokens from a repository.  The documents are expected to be handed to an output connector
(described elsewhere) for ingestion into some other back-end repository.

As is the case with all connectors under the LCF umbrella, an output connector consists of
two parts:

* A class implementing an interface (in this case, _org.apache.lcf.crawler.interfaces.IRepositoryConnector_)
* A set of JSP's that implement the crawler UI for the connector

h3. Implementing the Repository Connector class

A very good place to start is to read the javadoc for the repository connector interface.
 You will note that the javadoc describes the usage and pooling model for a connector class
pretty thoroughly.  It is very important to understand the model thoroughly in order to write
reliable connectors!  Use of static variables, for one thing, must be done in a very careful
way, to avoid issues that would be hard to detect with a cursory test.

The second thing to do is to examine some of the provided repository connector implementations.
 There are a wide variety of connectors include with LCF that exercise just about every aspect
of the repository connector interface.  These are:

* Documentum (uses RMI to segregate native code, etc.)
* FileNet (also uses RMI, but because it is picky about its open-source jar versions)
* File system (a good, but simple, example)
* LiveLink (demonstrates use of local keystore infrastructure)
* Memex
* Meridio (local keystore, web services, result sets)
* SharePoint (local keystore, web services)
* RSS (local keystore, binning)
* Web (local database schema, local keystore, binning, events and prerequisites)

You will also note that all of these connectors extend a framework-provided repository connector
base class, found at _org.apache.lcf.crawler.connectors.BaseRepositoryConnector_.  This base
class furnishes some basic bookkeeping logic for managing the connector pool, as well as default
implementations of some of the less typical functionality a connector may have.  For example,
connectors are allowed to have database tables of their own, which are instantiated when the
connector is registered, and are torn down when the connector is removed.  This is, however,
not very typical, and the base implementation reflects that.

TODO: More implementation details

h3. Implementing a set of Repository Connector JSPs

The repository connector class you write provides, through one of its methods, a symbolic
name where the crawler UI will look for repository connector UI components.  Your components
will therefore have the following path, relative to the crawler UI web application:

_connectors/<connector_symbolic_name>_

For a repository connector, you need to furnish the following JSPs:

|| JSP name || Where it fits ||
| headerconfig.jsp | Called during the header section of repository connector configuration
editing page |
| editconfig.jsp | Called during the body section of the repository connector configuration
editing page |
| postconfig.jsp | Called when configuration editing page is posted, either on a repost or
on a save |
| viewconfig.jsp | Called when the connection configuration is being viewed |
| headerspec.jsp | Called during the header section of a job definition editing page, for
which this repository connector has been selected |
| editspec.jsp | Called during the body section of a job definition editing page, for which
this repository connector has been selected |
| postspec.jsp | Called whenever a job definition that uses this repository connector is posted,
either for a repost or a save |
| viewspec.jsp | Called when a job definition that uses this repository connector is viewed
|

TODO: More implementation details


Change your notification preferences: http://cwiki.apache.org/confluence/users/viewnotifications.action

Mime
View raw message