manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CONNECTORS-988) ManifoldCF pipeline should not dump small files/streams to disk
Date Tue, 01 Jul 2014 08:30:25 GMT
Karl Wright created CONNECTORS-988:
--------------------------------------

             Summary: ManifoldCF pipeline should not dump small files/streams to disk
                 Key: CONNECTORS-988
                 URL: https://issues.apache.org/jira/browse/CONNECTORS-988
             Project: ManifoldCF
          Issue Type: Improvement
          Components: Framework agents process
    Affects Versions: ManifoldCF 1.7
            Reporter: Karl Wright
            Assignee: Karl Wright
             Fix For: ManifoldCF 1.7


The ManifoldCF pipeline, when it splits, dumps data to disk.  Since most files are small,
it should be more intelligent about this, and keep data in memory up to a certain size.

Turns out that TempFileInput and TempFileCharacterInput already support this functionality,
so just changing over to support these core constructs would do the trick.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message