kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5475) Connector config validation REST API endpoint not including fields for transformations
Date Wed, 21 Jun 2017 18:05:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057941#comment-16057941
] 

ASF GitHub Bot commented on KAFKA-5475:
---------------------------------------

GitHub user ewencp opened a pull request:

    https://github.com/apache/kafka/pull/3399

    KAFKA-5475: Connector config validation should include fields for defined transformation
aliases

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ewencp/kafka kafka-5475-validation-transformations

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/3399.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3399
    
----
commit 792a4e03379ff470bc98698455bbe75767739383
Author: Ewen Cheslack-Postava <me@ewencp.org>
Date:   2017-06-21T06:18:41Z

    KAFKA-5475: Connector config validation should include fields for defined transformation
aliases

----


> Connector config validation REST API endpoint not including fields for transformations
> --------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5475
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5475
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: Ewen Cheslack-Postava
>            Priority: Blocker
>             Fix For: 0.11.0.0
>
>
> An issue with how embedded transformation configurations are included seems to have been
introduced during 0.11.0.0. We are no longer seeing the `transforms.<xform>.type` being
included in the validation output.
> {code}
>  curl -X PUT -H "Content-Type: application/json" --data '{"connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector",
"transforms": "foo,bar"}' http://localhost:8083/connector-plugins/org.apache.kafka.connect.file.FileStreamSourceConnector/config/validate
 | jq
>   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>                                  Dload  Upload   Total   Spent    Left  Speed
> 100  3428  100  3325  100   103   344k  10917 --:--:-- --:--:-- --:--:--  360k
> {
>   "name": "org.apache.kafka.connect.file.FileStreamSourceConnector",
>   "error_count": 1,
>   "groups": [
>     "Common",
>     "Transforms"
>   ],
>   "configs": [
>     {
>       "definition": {
>         "name": "value.converter",
>         "type": "CLASS",
>         "required": false,
>         "default_value": null,
>         "importance": "LOW",
>         "documentation": "Converter class used to convert between Kafka Connect format
and the serialized form that is written to Kafka. This controls the format of the values in
messages written to or read from Kafka, and since this is independent of connectors it allows
any connector to work with any serialization format. Examples of common formats include JSON
and Avro.",
>         "group": "Common",
>         "width": "SHORT",
>         "display_name": "Value converter class",
>         "dependents": [],
>         "order": 5
>       },
>       "value": {
>         "name": "value.converter",
>         "value": null,
>         "recommended_values": [],
>         "errors": [],
>         "visible": true
>       }
>     },
>     {
>       "definition": {
>         "name": "name",
>         "type": "STRING",
>         "required": true,
>         "default_value": null,
>         "importance": "HIGH",
>         "documentation": "Globally unique name to use for this connector.",
>         "group": "Common",
>         "width": "MEDIUM",
>         "display_name": "Connector name",
>         "dependents": [],
>         "order": 1
>       },
>       "value": {
>         "name": "name",
>         "value": null,
>         "recommended_values": [],
>         "errors": [
>           "Missing required configuration \"name\" which has no default value."
>         ],
>         "visible": true
>       }
>     },
>     {
>       "definition": {
>         "name": "tasks.max",
>         "type": "INT",
>         "required": false,
>         "default_value": "1",
>         "importance": "HIGH",
>         "documentation": "Maximum number of tasks to use for this connector.",
>         "group": "Common",
>         "width": "SHORT",
>         "display_name": "Tasks max",
>         "dependents": [],
>         "order": 3
>       },
>       "value": {
>         "name": "tasks.max",
>         "value": "1",
>         "recommended_values": [],
>         "errors": [],
>         "visible": true
>       }
>     },
>     {
>       "definition": {
>         "name": "connector.class",
>         "type": "STRING",
>         "required": true,
>         "default_value": null,
>         "importance": "HIGH",
>         "documentation": "Name or alias of the class for this connector. Must be a subclass
of org.apache.kafka.connect.connector.Connector. If the connector is org.apache.kafka.connect.file.FileStreamSinkConnector,
you can either specify this full name,  or use \"FileStreamSink\" or \"FileStreamSinkConnector\"
to make the configuration a bit shorter",
>         "group": "Common",
>         "width": "LONG",
>         "display_name": "Connector class",
>         "dependents": [],
>         "order": 2
>       },
>       "value": {
>         "name": "connector.class",
>         "value": "org.apache.kafka.connect.file.FileStreamSourceConnector",
>         "recommended_values": [],
>         "errors": [],
>         "visible": true
>       }
>     },
>     {
>       "definition": {
>         "name": "key.converter",
>         "type": "CLASS",
>         "required": false,
>         "default_value": null,
>         "importance": "LOW",
>         "documentation": "Converter class used to convert between Kafka Connect format
and the serialized form that is written to Kafka. This controls the format of the keys in
messages written to or read from Kafka, and since this is independent of connectors it allows
any connector to work with any serialization format. Examples of common formats include JSON
and Avro.",
>         "group": "Common",
>         "width": "SHORT",
>         "display_name": "Key converter class",
>         "dependents": [],
>         "order": 4
>       },
>       "value": {
>         "name": "key.converter",
>         "value": null,
>         "recommended_values": [],
>         "errors": [],
>         "visible": true
>       }
>     },
>     {
>       "definition": {
>         "name": "transforms",
>         "type": "LIST",
>         "required": false,
>         "default_value": null,
>         "importance": "LOW",
>         "documentation": "Aliases for the transformations to be applied to records.",
>         "group": "Transforms",
>         "width": "LONG",
>         "display_name": "Transforms",
>         "dependents": [],
>         "order": 6
>       },
>       "value": {
>         "name": "transforms",
>         "value": "foo,bar",
>         "recommended_values": [],
>         "errors": [],
>         "visible": true
>       }
>     }
>   ]
> }
> {code}
> In this example, we should have seen `transforms.foo.type` and `transforms.bar.type`
included as additional configs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message