manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "roel goovaerts (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CONNECTORS-1599) response code 401 still gets deleted with the setting "keep unreachable documents"
Date Thu, 11 Apr 2019 09:45:00 GMT
roel goovaerts created CONNECTORS-1599:
------------------------------------------

             Summary: response code 401 still gets deleted with the setting "keep unreachable
documents"
                 Key: CONNECTORS-1599
                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1599
             Project: ManifoldCF
          Issue Type: Bug
          Components: Web connector
    Affects Versions: ManifoldCF 2.12
            Reporter: roel goovaerts


Even with the "Hop count mode" set to "keep unreachable documents, 'for now' || forever"
manifold deletes documents for which it receives a 401 response code.

The documentation does not specify such a distinction as described above. Is there some information/configuration
that I'm missing? Is there a reasoning behind the guaranteed deletion of a 401?

Ideally, for our use-case, we would want to remove all documents that return 404, but keep
everything which is due the server not responding or the crawler being unauthenticated.

Is there a way to configure this in a more granular fashion?

Regards,
roel



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message