manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <>
Subject [jira] [Resolved] (CONNECTORS-993) Pipeline does not handle "no document" case properly
Date Wed, 16 Jul 2014 12:57:04 GMT


Karl Wright resolved CONNECTORS-993.

    Resolution: Fixed


> Pipeline does not handle "no document" case properly
> ----------------------------------------------------
>                 Key: CONNECTORS-993
>                 URL:
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Framework agents process
>    Affects Versions: ManifoldCF 1.7
>            Reporter: Karl Wright
>            Assignee: Karl Wright
>             Fix For: ManifoldCF 1.7
> The incremental ingester and its pipeline code, and indeed transformation connectors
themselves, do not handle the "no document" case properly.
> What should happen is that whenever a document is not retransmitted into a branch of
the pipeline, the document version should nevertheless be recorded in the ingeststatus table
for the corresponding output.  But transformation connectors can interfere with that transmission
if they decide simply to not index an incoming document.  They may also receive a null RepositoryDocument
into the transformation method, and will be unequipped to handle that.
> Instead, I propose the following:
> (1) An explicit IIncrementalIngester method needs to be invented for handling this case
> (2) The activity class used for transformation methods should have a "noDocument()" method
> (3) IncrementalIngester needs to propagate "noDocument" method calls down the pipeline
without calling transformations on them

This message was sent by Atlassian JIRA

View raw message