kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-5475) Connector config validation REST API endpoint not including fields for transformations
Date Mon, 19 Jun 2017 23:03:00 GMT
Ewen Cheslack-Postava created KAFKA-5475:
--------------------------------------------

             Summary: Connector config validation REST API endpoint not including fields for
transformations
                 Key: KAFKA-5475
                 URL: https://issues.apache.org/jira/browse/KAFKA-5475
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
            Reporter: Ewen Cheslack-Postava
            Priority: Blocker
             Fix For: 0.11.0.0


An issue with how embedded transformation configurations are included seems to have been introduced
during 0.11.0.0. We are no longer seeing the `transforms.<xform>.type` being included
in the validation output.

{code}
 curl -X PUT -H "Content-Type: application/json" --data '{"connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector",
"transforms": "foo,bar"}' http://localhost:8083/connector-plugins/org.apache.kafka.connect.file.FileStreamSourceConnector/config/validate
 | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3428  100  3325  100   103   344k  10917 --:--:-- --:--:-- --:--:--  360k
{
  "name": "org.apache.kafka.connect.file.FileStreamSourceConnector",
  "error_count": 1,
  "groups": [
    "Common",
    "Transforms"
  ],
  "configs": [
    {
      "definition": {
        "name": "value.converter",
        "type": "CLASS",
        "required": false,
        "default_value": null,
        "importance": "LOW",
        "documentation": "Converter class used to convert between Kafka Connect format and
the serialized form that is written to Kafka. This controls the format of the values in messages
written to or read from Kafka, and since this is independent of connectors it allows any connector
to work with any serialization format. Examples of common formats include JSON and Avro.",
        "group": "Common",
        "width": "SHORT",
        "display_name": "Value converter class",
        "dependents": [],
        "order": 5
      },
      "value": {
        "name": "value.converter",
        "value": null,
        "recommended_values": [],
        "errors": [],
        "visible": true
      }
    },
    {
      "definition": {
        "name": "name",
        "type": "STRING",
        "required": true,
        "default_value": null,
        "importance": "HIGH",
        "documentation": "Globally unique name to use for this connector.",
        "group": "Common",
        "width": "MEDIUM",
        "display_name": "Connector name",
        "dependents": [],
        "order": 1
      },
      "value": {
        "name": "name",
        "value": null,
        "recommended_values": [],
        "errors": [
          "Missing required configuration \"name\" which has no default value."
        ],
        "visible": true
      }
    },
    {
      "definition": {
        "name": "tasks.max",
        "type": "INT",
        "required": false,
        "default_value": "1",
        "importance": "HIGH",
        "documentation": "Maximum number of tasks to use for this connector.",
        "group": "Common",
        "width": "SHORT",
        "display_name": "Tasks max",
        "dependents": [],
        "order": 3
      },
      "value": {
        "name": "tasks.max",
        "value": "1",
        "recommended_values": [],
        "errors": [],
        "visible": true
      }
    },
    {
      "definition": {
        "name": "connector.class",
        "type": "STRING",
        "required": true,
        "default_value": null,
        "importance": "HIGH",
        "documentation": "Name or alias of the class for this connector. Must be a subclass
of org.apache.kafka.connect.connector.Connector. If the connector is org.apache.kafka.connect.file.FileStreamSinkConnector,
you can either specify this full name,  or use \"FileStreamSink\" or \"FileStreamSinkConnector\"
to make the configuration a bit shorter",
        "group": "Common",
        "width": "LONG",
        "display_name": "Connector class",
        "dependents": [],
        "order": 2
      },
      "value": {
        "name": "connector.class",
        "value": "org.apache.kafka.connect.file.FileStreamSourceConnector",
        "recommended_values": [],
        "errors": [],
        "visible": true
      }
    },
    {
      "definition": {
        "name": "key.converter",
        "type": "CLASS",
        "required": false,
        "default_value": null,
        "importance": "LOW",
        "documentation": "Converter class used to convert between Kafka Connect format and
the serialized form that is written to Kafka. This controls the format of the keys in messages
written to or read from Kafka, and since this is independent of connectors it allows any connector
to work with any serialization format. Examples of common formats include JSON and Avro.",
        "group": "Common",
        "width": "SHORT",
        "display_name": "Key converter class",
        "dependents": [],
        "order": 4
      },
      "value": {
        "name": "key.converter",
        "value": null,
        "recommended_values": [],
        "errors": [],
        "visible": true
      }
    },
    {
      "definition": {
        "name": "transforms",
        "type": "LIST",
        "required": false,
        "default_value": null,
        "importance": "LOW",
        "documentation": "Aliases for the transformations to be applied to records.",
        "group": "Transforms",
        "width": "LONG",
        "display_name": "Transforms",
        "dependents": [],
        "order": 6
      },
      "value": {
        "name": "transforms",
        "value": "foo,bar",
        "recommended_values": [],
        "errors": [],
        "visible": true
      }
    }
  ]
}
{code}

In this example, we should have seen `transforms.foo.type` and `transforms.bar.type` included
as additional configs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message