manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CONNECTORS-1567) export of web connection bandwidth throttling
Date Tue, 08 Jan 2019 22:07:00 GMT

    [ https://issues.apache.org/jira/browse/CONNECTORS-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737581#comment-16737581
] 

Karl Wright commented on CONNECTORS-1567:
-----------------------------------------

Reading and writing sides match.

In XML, the format would look like this:

{code}
<connection>
  ...
  <throttle>
    <match>match_value</match>
    <description>description</description>
    <rate>rate_value</rate>
  </throttle>
</connection>
{code}

This gets translated to JSON, which should merge the "throttle" fields into one throttle array,
like this:

{code}
throttle: [ {... first throttle ... }, {... second throttle ... } ...]
{code}

That's obviously not happening and I need to figure out why.


> export of web connection bandwidth throttling
> ---------------------------------------------
>
>                 Key: CONNECTORS-1567
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1567
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Web connector
>    Affects Versions: ManifoldCF 2.11, ManifoldCF 2.12
>            Reporter: Tim Steenbeke
>            Assignee: Karl Wright
>            Priority: Major
>             Fix For: ManifoldCF 2.13
>
>
> When exporting the web connector using the API, it doesn't export the bandwidth throttling.
>  Than when importing this connector to a clean manifoldcf it creates the connector with
basic bandwidth.
>  When using the connector in a job it works properly.
> The issue here is that the connector isn't created with correct bandwidth throttling.
>  And the connector gives issues in the UI when trying to view or edit.
> (related to issue: [CONNECTORS-1568|https://issues.apache.org/jira/projects/CONNECTORS/issues/CONNECTORS-1568])
> e.g.:
> {code:java}
> {
>   "name": "test_web",
>   "configuration": null,
>     "_PARAMETER_": [
>       {
>         "_attribute_name": "Email address",
>         "_value_": "tim.steenbeke@formica.digital"
>       },
>       {
>         "_attribute_name": "Robots usage",
>         "_value_": "all"
>       },
>       {
>         "_attribute_name": "Meta robots tags usage",
>         "_value_": "all"
>       },
>       {
>         "_attribute_name": "Proxy host",
>         "_value_": ""
>       },
>       {
>         "_attribute_name": "Proxy port",
>         "_value_": ""
>       },
>       {
>         "_attribute_name": "Proxy authentication domain",
>         "_value_": ""
>       },
>       {
>         "_attribute_name": "Proxy authentication user name",
>         "_value_": ""
>       },
>       {
>         "_attribute_name": "Proxy authentication password",
>         "_value_": ""
>       }
>     ]
>   },
>   "description": "Website repository standard settup",
>   "throttle": null,
>   "max_connections": 10,
>   "class_name": "org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector",
>   "acl_authority": null
> }{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message