lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Harness (Updated) (JIRA)" <>
Subject [jira] [Updated] (SOLR-2851) Allow Data Import Handler to Archive Files After Processing
Date Tue, 25 Oct 2011 12:38:32 GMT


Josh Harness updated SOLR-2851:


Custom file data source that archives files when the close method on the input stream reader
is called. Java package name is still specific to my company and needs to be modified to reflect
the apache structure. Also, tests are needed. 
> Allow Data Import Handler to Archive Files After Processing
> -----------------------------------------------------------
>                 Key: SOLR-2851
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>          Components: contrib - DataImportHandler
>    Affects Versions: 3.3
>            Reporter: Josh Harness
>            Priority: Minor
>              Labels: dataimport, solr
>             Fix For: 3.3
>         Attachments:
> It would be great if, after processing flat files, DIH could place them in an archive
folder so that other system processes could compress them, delete them, etc. We needed this
feature, so I extended FileDataSource and decorated the InputStreamReader such that when the
close method is called, the file is moved out of the way. This concrete implementation did
exactly what I needed. However, a more correct textbook approach would involve refactoring
FileListEntityProcessor such that it could take a configurable list of post-processing action
policy/command classes so that the user could provide custom behavior that suits nearly any
scenario (in a Chain of Responsibility style pattern). Nonetheless, I wanted to provide the
suggestion, a basic one-off concrete implementation of archiving and start the discussion
for where things might head in the future. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message