From commits-return-10718-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Sat Nov 10 01:11:40 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id D9954180627 for ; Sat, 10 Nov 2018 01:11:34 +0100 (CET) Received: (qmail 21654 invoked by uid 500); 10 Nov 2018 00:11:33 -0000 Mailing-List: contact commits-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list commits@kafka.apache.org Received: (qmail 21645 invoked by uid 99); 10 Nov 2018 00:11:33 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Nov 2018 00:11:33 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id B56AC870E7; Sat, 10 Nov 2018 00:11:32 +0000 (UTC) Date: Sat, 10 Nov 2018 00:11:32 +0000 To: "commits@kafka.apache.org" Subject: [kafka-site] branch asf-site updated: Add generated configs for Kafka 2.1.0 RC1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <154180869263.25439.6084572135308559964@gitbox.apache.org> From: lindong@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: kafka-site X-Git-Refname: refs/heads/asf-site X-Git-Reftype: branch X-Git-Oldrev: d48694d81911ba802f3ea008c3ed72e071e651ac X-Git-Newrev: 072afe8317005ef577dff660220e49609f875de7 X-Git-Rev: 072afe8317005ef577dff660220e49609f875de7 X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. lindong pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/kafka-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 072afe8 Add generated configs for Kafka 2.1.0 RC1 072afe8 is described below commit 072afe8317005ef577dff660220e49609f875de7 Author: Dong Lin AuthorDate: Fri Nov 9 16:10:24 2018 -0800 Add generated configs for Kafka 2.1.0 RC1 --- 21/generated/admin_client_config.html | 102 + 21/generated/connect_config.html | 176 + 21/generated/connect_metrics.html | 181 + 21/generated/connect_transforms.html | 228 + 21/generated/consumer_config.html | 140 + 21/generated/consumer_metrics.html | 79 + 21/generated/kafka_config.html | 392 ++ 21/generated/producer_config.html | 134 + 21/generated/producer_metrics.html | 78 + 21/generated/protocol_api_keys.html | 91 + 21/generated/protocol_errors.html | 86 + 21/generated/protocol_messages.html | 7281 +++++++++++++++++++++++++++++ 21/generated/protocol_types.html | 20 + 21/generated/sink_connector_config.html | 46 + 21/generated/source_connector_config.html | 36 + 21/generated/streams_config.html | 88 + 21/generated/topic_config.html | 61 + 17 files changed, 9219 insertions(+) diff --git a/21/generated/admin_client_config.html b/21/generated/admin_client_config.html index e69de29..2f20d86 100644 --- a/21/generated/admin_client_config.html +++ b/21/generated/admin_client_config.html @@ -0,0 +1,102 @@ + + + + + + + + + + + + + + + + + + + + + + [...] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
bootstrap.serversA list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form host1:port1,host2:port2,.... Since these servers are just used for the initial connection to discover the full cluster me [...] +
ssl.key.passwordThe password of the private key in the key store file. This is optional for client.passwordnullhigh
ssl.keystore.locationThe location of the key store file. This is optional for client and can be used for two-way authentication for client.stringnullhigh
ssl.keystore.passwordThe store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured. passwordnullhigh
ssl.truststore.locationThe location of the trust store file. stringnullhigh
ssl.truststore.passwordThe password for the trust store file. If a password is not set access to the truststore is still available, but integrity checking is disabled.passwordnullhigh
client.dns.lookup

Controls how the client uses DNS lookups.

If set to use_all_dns_ips then, when the lookup returns multiple IP addresses for a hostname, they will all be attempted to connect to before failing the connection. Applies to both bootstrap and advertised servers.

If the value is resolve_canonical_bootstrap_servers_only each entry will be resolved and expanded into a list of canonical names.

string
client.idAn id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.string""medium
connections.max.idle.msClose idle connections after the number of milliseconds specified by this config.long300000medium
receive.buffer.bytesThe size of the TCP receive buffer (SO_RCVBUF) to use when reading data. If the value is -1, the OS default will be used.int65536[-1,...]medium
request.timeout.msThe configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.int120000[0,...]medium
sasl.client.callback.handler.classThe fully qualified name of a SASL client callback handler class that implements the AuthenticateCallbackHandler interface.classnullmedium
sasl.jaas.configJAAS login context parameters for SASL connections in the format used by JAAS configuration files. JAAS configuration file format is described here. The format for the value is: 'loginModuleClass controlFlag (optionName=optionValue)*;'. For brokers, the config must be prefixed with listener prefix and SASL mechanism name in lower- [...] +
sasl.kerberos.service.nameThe Kerberos principal name that Kafka runs as. This can be defined either in Kafka's JAAS config or in Kafka's config.stringnullmedium
sasl.login.callback.handler.classThe fully qualified name of a SASL login callback handler class that implements the AuthenticateCallbackHandler interface. For brokers, login callback handler config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.callback.handler.class=com.example.CustomScramLoginCallbackHandlerclassnullmedium [...] +
sasl.login.classThe fully qualified name of a class that implements the Login interface. For brokers, login config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.class=com.example.CustomScramLoginclassnullmedium
sasl.mechanismSASL mechanism used for client connections. This may be any mechanism for which a security provider is available. GSSAPI is the default mechanism.stringGSSAPImedium
security.protocolProtocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.stringPLAINTEXTmedium
send.buffer.bytesThe size of the TCP send buffer (SO_SNDBUF) to use when sending data. If the value is -1, the OS default will be used.int131072[-1,...]medium
ssl.enabled.protocolsThe list of protocols enabled for SSL connections.listTLSv1.2,TLSv1.1,TLSv1medium
ssl.keystore.typeThe file format of the key store file. This is optional for client.stringJKSmedium
ssl.protocolThe SSL protocol used to generate the SSLContext. Default setting is TLS, which is fine for most cases. Allowed values in recent JVMs are TLS, TLSv1.1 and TLSv1.2. SSL, SSLv2 and SSLv3 may be supported in older JVMs, but their usage is discouraged due to known security vulnerabilities.stringTLSmedium
ssl.providerThe name of the security provider used for SSL connections. Default value is the default security provider of the JVM.stringnullmedium
ssl.truststore.typeThe file format of the trust store file.stringJKSmedium
metadata.max.age.msThe period of time in milliseconds after which we force a refresh of metadata even if we haven't seen any partition leadership changes to proactively discover any new brokers or partitions.long300000[0,...]low
metric.reportersA list of classes to use as metrics reporters. Implementing the org.apache.kafka.common.metrics.MetricsReporter interface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics.list""low
metrics.num.samplesThe number of samples maintained to compute metrics.int2[1,...]low
metrics.recording.levelThe highest recording level for metrics.stringINFO[INFO, DEBUG]low
metrics.sample.window.msThe window of time a metrics sample is computed over.long30000[0,...]low
reconnect.backoff.max.msThe maximum amount of time in milliseconds to wait when reconnecting to a broker that has repeatedly failed to connect. If provided, the backoff per host will increase exponentially for each consecutive connection failure, up to this maximum. After calculating the backoff increase, 20% random jitter is added to avoid connection storms.long1000[0,...]low
reconnect.backoff.msThe base amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop. This backoff applies to all connection attempts by the client to a broker.long50[0,...]low
retriesSetting a value greater than zero will cause the client to resend any request that fails with a potentially transient error.int5[0,...]low
retry.backoff.msThe amount of time to wait before attempting to retry a failed request. This avoids repeatedly sending requests in a tight loop under some failure scenarios.long100[0,...]low
sasl.kerberos.kinit.cmdKerberos kinit command path.string/usr/bin/kinitlow
sasl.kerberos.min.time.before.reloginLogin thread sleep time between refresh attempts.long60000low
sasl.kerberos.ticket.renew.jitterPercentage of random jitter added to the renewal time.double0.05low
sasl.kerberos.ticket.renew.window.factorLogin thread will sleep until the specified window factor of time from last refresh to ticket's expiry has been reached, at which time it will try to renew the ticket.double0.8low
sasl.login.refresh.buffer.secondsThe amount of buffer time before credential expiration to maintain when refreshing a credential, in seconds. If a refresh would otherwise occur closer to expiration than the number of buffer seconds then the refresh will be moved up to maintain as much of the buffer time as possible. Legal values are between 0 and 3600 (1 hour); a default value of 300 (5 minutes) is used if no value is specified. This value and sasl.login.refresh.min.pe [...] +
sasl.login.refresh.min.period.secondsThe desired minimum time for the login refresh thread to wait before refreshing a credential, in seconds. Legal values are between 0 and 900 (15 minutes); a default value of 60 (1 minute) is used if no value is specified. This value and sasl.login.refresh.buffer.seconds are both ignored if their sum exceeds the remaining lifetime of a credential. Currently applies only to OAUTHBEARER.short60 +sasl.login.refresh.window.factorLogin refresh thread will sleep until the specified window factor relative to the credential's lifetime has been reached, at which time it will try to refresh the credential. Legal values are between 0.5 (50%) and 1.0 (100%) inclusive; a default value of 0.8 (80%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.8[0.5,...,1.0]low
sasl.login.refresh.window.jitterThe maximum amount of random jitter relative to the credential's lifetime that is added to the login refresh thread's sleep time. Legal values are between 0 and 0.25 (25%) inclusive; a default value of 0.05 (5%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.05[0.0,...,0.25]low
ssl.cipher.suitesA list of cipher suites. This is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. By default all the available cipher suites are supported.listnulllow
ssl.endpoint.identification.algorithmThe endpoint identification algorithm to validate server hostname using server certificate. stringhttpslow
ssl.keymanager.algorithmThe algorithm used by key manager factory for SSL connections. Default value is the key manager factory algorithm configured for the Java Virtual Machine.stringSunX509low
ssl.secure.random.implementationThe SecureRandom PRNG implementation to use for SSL cryptography operations. stringnulllow
ssl.trustmanager.algorithmThe algorithm used by trust manager factory for SSL connections. Default value is the trust manager factory algorithm configured for the Java Virtual Machine.stringPKIXlow
diff --git a/21/generated/connect_config.html b/21/generated/connect_config.html index e69de29..e24e6ad 100644 --- a/21/generated/connect_config.html +++ b/21/generated/connect_config.html @@ -0,0 +1,176 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + [...] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
config.storage.topicThe name of the Kafka topic where connector configurations are storedstringhigh
group.idA unique string that identifies the Connect cluster group this worker belongs to.stringhigh
key.converterConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the keys in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro.classhigh
offset.storage.topicThe name of the Kafka topic where connector offsets are storedstringhigh
status.storage.topicThe name of the Kafka topic where connector and task status are storedstringhigh
value.converterConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the values in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro.classhigh
bootstrap.serversA list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form host1:port1,host2:port2,.... Since these servers are just used for the initial connection to discover the full cluster me [...] +
heartbeat.interval.msThe expected time between heartbeats to the group coordinator when using Kafka's group management facilities. Heartbeats are used to ensure that the worker's session stays active and to facilitate rebalancing when new members join or leave the group. The value must be set lower than session.timeout.ms, but typically should be set no higher than 1/3 of that value. It can be adjusted even lower to control the expected time for normal reba [...] +
rebalance.timeout.msThe maximum allowed time for each worker to join the group once a rebalance has begun. This is basically a limit on the amount of time needed for all tasks to flush any pending data and commit offsets. If the timeout is exceeded, then the worker will be removed from the group, which will cause offset commit failures.int60000high
session.timeout.msThe timeout used to detect worker failures. The worker sends periodic heartbeats to indicate its liveness to the broker. If no heartbeats are received by the broker before the expiration of this session timeout, then the broker will remove the worker from the group and initiate a rebalance. Note that the value must be in the allowable range as configured in the broker configuration by group.min.session.timeout.ms and group.max.sessio [...] +
ssl.key.passwordThe password of the private key in the key store file. This is optional for client.passwordnullhigh
ssl.keystore.locationThe location of the key store file. This is optional for client and can be used for two-way authentication for client.stringnullhigh
ssl.keystore.passwordThe store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured. passwordnullhigh
ssl.truststore.locationThe location of the trust store file. stringnullhigh
ssl.truststore.passwordThe password for the trust store file. If a password is not set access to the truststore is still available, but integrity checking is disabled.passwordnullhigh
client.dns.lookup

Controls how the client uses DNS lookups.

If set to use_all_dns_ips then, when the lookup returns multiple IP addresses for a hostname, they will all be attempted to connect to before failing the connection. Applies to both bootstrap and advertised servers.

If the value is resolve_canonical_bootstrap_servers_only each entry will be resolved and expanded into a list of canonical names.

string
connections.max.idle.msClose idle connections after the number of milliseconds specified by this config.long540000medium
receive.buffer.bytesThe size of the TCP receive buffer (SO_RCVBUF) to use when reading data. If the value is -1, the OS default will be used.int32768[0,...]medium
request.timeout.msThe configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.int40000[0,...]medium
sasl.client.callback.handler.classThe fully qualified name of a SASL client callback handler class that implements the AuthenticateCallbackHandler interface.classnullmedium
sasl.jaas.configJAAS login context parameters for SASL connections in the format used by JAAS configuration files. JAAS configuration file format is described here. The format for the value is: 'loginModuleClass controlFlag (optionName=optionValue)*;'. For brokers, the config must be prefixed with listener prefix and SASL mechanism name in lower- [...] +
sasl.kerberos.service.nameThe Kerberos principal name that Kafka runs as. This can be defined either in Kafka's JAAS config or in Kafka's config.stringnullmedium
sasl.login.callback.handler.classThe fully qualified name of a SASL login callback handler class that implements the AuthenticateCallbackHandler interface. For brokers, login callback handler config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.callback.handler.class=com.example.CustomScramLoginCallbackHandlerclassnullmedium [...] +
sasl.login.classThe fully qualified name of a class that implements the Login interface. For brokers, login config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.class=com.example.CustomScramLoginclassnullmedium
sasl.mechanismSASL mechanism used for client connections. This may be any mechanism for which a security provider is available. GSSAPI is the default mechanism.stringGSSAPImedium
security.protocolProtocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.stringPLAINTEXTmedium
send.buffer.bytesThe size of the TCP send buffer (SO_SNDBUF) to use when sending data. If the value is -1, the OS default will be used.int131072[0,...]medium
ssl.enabled.protocolsThe list of protocols enabled for SSL connections.listTLSv1.2,TLSv1.1,TLSv1medium
ssl.keystore.typeThe file format of the key store file. This is optional for client.stringJKSmedium
ssl.protocolThe SSL protocol used to generate the SSLContext. Default setting is TLS, which is fine for most cases. Allowed values in recent JVMs are TLS, TLSv1.1 and TLSv1.2. SSL, SSLv2 and SSLv3 may be supported in older JVMs, but their usage is discouraged due to known security vulnerabilities.stringTLSmedium
ssl.providerThe name of the security provider used for SSL connections. Default value is the default security provider of the JVM.stringnullmedium
ssl.truststore.typeThe file format of the trust store file.stringJKSmedium
worker.sync.timeout.msWhen the worker is out of sync with other workers and needs to resynchronize configurations, wait up to this amount of time before giving up, leaving the group, and waiting a backoff period before rejoining.int3000medium
worker.unsync.backoff.msWhen the worker is out of sync with other workers and fails to catch up within worker.sync.timeout.ms, leave the Connect cluster for this long before rejoining.int300000medium
access.control.allow.methodsSets the methods supported for cross origin requests by setting the Access-Control-Allow-Methods header. The default value of the Access-Control-Allow-Methods header allows cross origin requests for GET, POST and HEAD.string""low
access.control.allow.originValue to set the Access-Control-Allow-Origin header to for REST API requests.To enable cross origin access, set this to the domain of the application that should be permitted to access the API, or '*' to allow access from any domain. The default value only allows access from the domain of the REST API.string""low
client.idAn id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.string""low
config.providersComma-separated names of ConfigProvider classes, loaded and used in the order specified. Implementing the interface ConfigProvider allows you to replace variable references in connector configurations, such as for externalized secrets. list""low
config.storage.replication.factorReplication factor used when creating the configuration storage topicshort3[1,...]low
header.converterHeaderConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the header values in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro. By default, the SimpleHeaderConverter is used to serialize header values to strings and deseria [...] +
internal.key.converterConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the keys in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro. This setting controls the format used for internal bookkeeping data used by the framework, such as confi [...] +
internal.value.converterConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the values in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro. This setting controls the format used for internal bookkeeping data used by the framework, such as c [...] +
listenersList of comma-separated URIs the REST API will listen on. The supported protocols are HTTP and HTTPS. + Specify hostname as 0.0.0.0 to bind to all interfaces. + Leave hostname empty to bind to default interface. + Examples of legal listener lists: HTTP://myhost:8083,HTTPS://myhost:8084listnulllow
metadata.max.age.msThe period of time in milliseconds after which we force a refresh of metadata even if we haven't seen any partition leadership changes to proactively discover any new brokers or partitions.long300000[0,...]low
metric.reportersA list of classes to use as metrics reporters. Implementing the org.apache.kafka.common.metrics.MetricsReporter interface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics.list""low
metrics.num.samplesThe number of samples maintained to compute metrics.int2[1,...]low
metrics.recording.levelThe highest recording level for metrics.stringINFO[INFO, DEBUG]low
metrics.sample.window.msThe window of time a metrics sample is computed over.long30000[0,...]low
offset.flush.interval.msInterval at which to try committing offsets for tasks.long60000low
offset.flush.timeout.msMaximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt.long5000low
offset.storage.partitionsThe number of partitions used when creating the offset storage topicint25[1,...]low
offset.storage.replication.factorReplication factor used when creating the offset storage topicshort3[1,...]low
plugin.pathList of paths separated by commas (,) that contain plugins (connectors, converters, transformations). The list should consist of top level directories that include any combination of: +a) directories immediately containing jars with plugins and their dependencies +b) uber-jars with plugins and their dependencies +c) directories immediately containing the package directory structure of classes of plugins and their dependencies +Note: symlinks will be followed to discover dependencies or plugins. +Examples: plugin.path=/usr/local/share/java,/usr/local/share/kafka/plugins,/opt/connectorslistnulllow
reconnect.backoff.max.msThe maximum amount of time in milliseconds to wait when reconnecting to a broker that has repeatedly failed to connect. If provided, the backoff per host will increase exponentially for each consecutive connection failure, up to this maximum. After calculating the backoff increase, 20% random jitter is added to avoid connection storms.long1000[0,...]low
reconnect.backoff.msThe base amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop. This backoff applies to all connection attempts by the client to a broker.long50[0,...]low
rest.advertised.host.nameIf this is set, this is the hostname that will be given out to other workers to connect to.stringnulllow
rest.advertised.listenerSets the advertised listener (HTTP or HTTPS) which will be given to other workers to use.stringnulllow
rest.advertised.portIf this is set, this is the port that will be given out to other workers to connect to.intnulllow
rest.extension.classesComma-separated names of ConnectRestExtension classes, loaded and called in the order specified. Implementing the interface ConnectRestExtension allows you to inject into Connect's REST API user defined resources like filters. Typically used to add custom capability like logging, security, etc. list""low
rest.host.nameHostname for the REST API. If this is set, it will only bind to this interface.stringnulllow
rest.portPort for the REST API to listen on.int8083low
retry.backoff.msThe amount of time to wait before attempting to retry a failed request to a given topic partition. This avoids repeatedly sending requests in a tight loop under some failure scenarios.long100[0,...]low
sasl.kerberos.kinit.cmdKerberos kinit command path.string/usr/bin/kinitlow
sasl.kerberos.min.time.before.reloginLogin thread sleep time between refresh attempts.long60000low
sasl.kerberos.ticket.renew.jitterPercentage of random jitter added to the renewal time.double0.05low
sasl.kerberos.ticket.renew.window.factorLogin thread will sleep until the specified window factor of time from last refresh to ticket's expiry has been reached, at which time it will try to renew the ticket.double0.8low
sasl.login.refresh.buffer.secondsThe amount of buffer time before credential expiration to maintain when refreshing a credential, in seconds. If a refresh would otherwise occur closer to expiration than the number of buffer seconds then the refresh will be moved up to maintain as much of the buffer time as possible. Legal values are between 0 and 3600 (1 hour); a default value of 300 (5 minutes) is used if no value is specified. This value and sasl.login.refresh.min.pe [...] +
sasl.login.refresh.min.period.secondsThe desired minimum time for the login refresh thread to wait before refreshing a credential, in seconds. Legal values are between 0 and 900 (15 minutes); a default value of 60 (1 minute) is used if no value is specified. This value and sasl.login.refresh.buffer.seconds are both ignored if their sum exceeds the remaining lifetime of a credential. Currently applies only to OAUTHBEARER.short60 +sasl.login.refresh.window.factorLogin refresh thread will sleep until the specified window factor relative to the credential's lifetime has been reached, at which time it will try to refresh the credential. Legal values are between 0.5 (50%) and 1.0 (100%) inclusive; a default value of 0.8 (80%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.8[0.5,...,1.0]low
sasl.login.refresh.window.jitterThe maximum amount of random jitter relative to the credential's lifetime that is added to the login refresh thread's sleep time. Legal values are between 0 and 0.25 (25%) inclusive; a default value of 0.05 (5%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.05[0.0,...,0.25]low
ssl.cipher.suitesA list of cipher suites. This is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. By default all the available cipher suites are supported.listnulllow
ssl.client.authConfigures kafka broker to request client authentication. The following settings are common:
  • ssl.client.auth=required If set to required client authentication is required.
  • ssl.client.auth=requested This means client authentication is optional. unlike requested , if this option is set client can choose not to provide authentication information about itself
  • ssl.client.auth=none This means client authe [...] +
ssl.endpoint.identification.algorithmThe endpoint identification algorithm to validate server hostname using server certificate. stringhttpslow
ssl.keymanager.algorithmThe algorithm used by key manager factory for SSL connections. Default value is the key manager factory algorithm configured for the Java Virtual Machine.stringSunX509low
ssl.secure.random.implementationThe SecureRandom PRNG implementation to use for SSL cryptography operations. stringnulllow
ssl.trustmanager.algorithmThe algorithm used by trust manager factory for SSL connections. Default value is the trust manager factory algorithm configured for the Java Virtual Machine.stringPKIXlow
status.storage.partitionsThe number of partitions used when creating the status storage topicint5[1,...]low
status.storage.replication.factorReplication factor used when creating the status storage topicshort3[1,...]low
task.shutdown.graceful.timeout.msAmount of time to wait for tasks to shutdown gracefully. This is the total amount of time, not per task. All task have shutdown triggered, then they are waited on sequentially.long5000low
diff --git a/21/generated/connect_metrics.html b/21/generated/connect_metrics.html index e69de29..8b5b27e 100644 --- a/21/generated/connect_metrics.html +++ b/21/generated/connect_metrics.html @@ -0,0 +1,181 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
kafka.connect:type=connect-worker-metrics
Attribute nameDescription
connector-countThe number of connectors run in this worker.
connector-startup-attempts-totalThe total number of connector startups that this worker has attempted.
connector-startup-failure-percentageThe average percentage of this worker's connectors starts that failed.
connector-startup-failure-totalThe total number of connector starts that failed.
connector-startup-success-percentageThe average percentage of this worker's connectors starts that succeeded.
connector-startup-success-totalThe total number of connector starts that succeeded.
task-countThe number of tasks run in this worker.
task-startup-attempts-totalThe total number of task startups that this worker has attempted.
task-startup-failure-percentageThe average percentage of this worker's tasks starts that failed.
task-startup-failure-totalThe total number of task starts that failed.
task-startup-success-percentageThe average percentage of this worker's tasks starts that succeeded.
task-startup-success-totalThe total number of task starts that succeeded.
kafka.connect:type=connect-worker-rebalance-metrics
Attribute nameDescription
completed-rebalances-totalThe total number of rebalances completed by this worker.
epochThe epoch or generation number of this worker.
leader-nameThe name of the group leader.
rebalance-avg-time-msThe average time in milliseconds spent by this worker to rebalance.
rebalance-max-time-msThe maximum time in milliseconds spent by this worker to rebalance.
rebalancingWhether this worker is currently rebalancing.
time-since-last-rebalance-msThe time in milliseconds since this worker completed the most recent rebalance.
kafka.connect:type=connector-metrics,connector="{connector}"
Attribute nameDescription
connector-classThe name of the connector class.
connector-typeThe type of the connector. One of 'source' or 'sink'.
connector-versionThe version of the connector class, as reported by the connector.
statusThe status of the connector. One of 'unassigned', 'running', 'paused', 'failed', or 'destroyed'.
kafka.connect:type=connector-task-metrics,connector="{connector}",task="{task}"
Attribute nameDescription
batch-size-avgThe average size of the batches processed by the connector.
batch-size-maxThe maximum size of the batches processed by the connector.
offset-commit-avg-time-msThe average time in milliseconds taken by this task to commit offsets.
offset-commit-failure-percentageThe average percentage of this task's offset commit attempts that failed.
offset-commit-max-time-msThe maximum time in milliseconds taken by this task to commit offsets.
offset-commit-success-percentageThe average percentage of this task's offset commit attempts that succeeded.
pause-ratioThe fraction of time this task has spent in the pause state.
running-ratioThe fraction of time this task has spent in the running state.
statusThe status of the connector task. One of 'unassigned', 'running', 'paused', 'failed', or 'destroyed'.
kafka.connect:type=sink-task-metrics,connector="{connector}",task="{task}"
Attribute nameDescription
offset-commit-completion-rateThe average per-second number of offset commit completions that were completed successfully.
offset-commit-completion-totalThe total number of offset commit completions that were completed successfully.
offset-commit-seq-noThe current sequence number for offset commits.
offset-commit-skip-rateThe average per-second number of offset commit completions that were received too late and skipped/ignored.
offset-commit-skip-totalThe total number of offset commit completions that were received too late and skipped/ignored.
partition-countThe number of topic partitions assigned to this task belonging to the named sink connector in this worker.
put-batch-avg-time-msThe average time taken by this task to put a batch of sinks records.
put-batch-max-time-msThe maximum time taken by this task to put a batch of sinks records.
sink-record-active-countThe number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task.
sink-record-active-count-avgThe average number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task.
sink-record-active-count-maxThe maximum number of records that have been read from Kafka but not yet completely committed/flushed/acknowledged by the sink task.
sink-record-lag-maxThe maximum lag in terms of number of records that the sink task is behind the consumer's position for any topic partitions.
sink-record-read-rateThe average per-second number of records read from Kafka for this task belonging to the named sink connector in this worker. This is before transformations are applied.
sink-record-read-totalThe total number of records read from Kafka by this task belonging to the named sink connector in this worker, since the task was last restarted.
sink-record-send-rateThe average per-second number of records output from the transformations and sent/put to this task belonging to the named sink connector in this worker. This is after transformations are applied and excludes any records filtered out by the transformations.
sink-record-send-totalThe total number of records output from the transformations and sent/put to this task belonging to the named sink connector in this worker, since the task was last restarted.
kafka.connect:type=source-task-metrics,connector="{connector}",task="{task}"
Attribute nameDescription
poll-batch-avg-time-msThe average time in milliseconds taken by this task to poll for a batch of source records.
poll-batch-max-time-msThe maximum time in milliseconds taken by this task to poll for a batch of source records.
source-record-active-countThe number of records that have been produced by this task but not yet completely written to Kafka.
source-record-active-count-avgThe average number of records that have been produced by this task but not yet completely written to Kafka.
source-record-active-count-maxThe maximum number of records that have been produced by this task but not yet completely written to Kafka.
source-record-poll-rateThe average per-second number of records produced/polled (before transformation) by this task belonging to the named source connector in this worker.
source-record-poll-totalThe total number of records produced/polled (before transformation) by this task belonging to the named source connector in this worker.
source-record-write-rateThe average per-second number of records output from the transformations and written to Kafka for this task belonging to the named source connector in this worker. This is after transformations are applied and excludes any records filtered out by the transformations.
source-record-write-totalThe number of records output from the transformations and written to Kafka for this task belonging to the named source connector in this worker, since the task was last restarted.
kafka.connect:type=task-error-metrics,connector="{connector}",task="{task}"
Attribute nameDescription
deadletterqueue-produce-failuresThe number of failed writes to the dead letter queue.
deadletterqueue-produce-requestsThe number of attempted writes to the dead letter queue.
last-error-timestampThe epoch timestamp when this task last encountered an error.
total-errors-loggedThe number of errors that were logged.
total-record-errorsThe number of record processing errors in this task.
total-record-failuresThe number of record processing failures in this task.
total-records-skippedThe number of records skipped due to errors.
total-retriesThe number of operations retried.
diff --git a/21/generated/connect_transforms.html b/21/generated/connect_transforms.html index e69de29..452ab01 100644 --- a/21/generated/connect_transforms.html +++ b/21/generated/connect_transforms.html @@ -0,0 +1,228 @@ +
+
org.apache.kafka.connect.transforms.InsertField
+Insert field(s) using attributes from the record metadata or a configured static value.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.InsertField$Key) or value (org.apache.kafka.connect.transforms.InsertField$Value). +

+ + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
offset.fieldField name for Kafka offset - only applicable to sink connectors.
Suffix with ! to make this a required field, or ? to keep it optional (the default).		stringnullmedium
partition.fieldField name for Kafka partition. Suffix with ! to make this a required field, or ? to keep it optional (the default).stringnullmedium
static.fieldField name for static data field. Suffix with ! to make this a required field, or ? to keep it optional (the default).stringnullmedium
static.valueStatic field value, if field name configured.stringnullmedium
timestamp.fieldField name for record timestamp. Suffix with ! to make this a required field, or ? to keep it optional (the default).stringnullmedium
topic.fieldField name for Kafka topic. Suffix with ! to make this a required field, or ? to keep it optional (the default).stringnullmedium
+

+
+
org.apache.kafka.connect.transforms.ReplaceField
+Filter or rename fields.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.ReplaceField$Key) or value (org.apache.kafka.connect.transforms.ReplaceField$Value). +

+ + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
blacklistFields to exclude. This takes precedence over the whitelist.list""medium
renamesField rename mappings.list""list of colon-delimited pairs, e.g. foo:bar,abc:xyzmedium
whitelistFields to include. If specified, only these fields will be used.list""medium
+

+
+
org.apache.kafka.connect.transforms.MaskField
+Mask specified fields with a valid null value for the field type (i.e. 0, false, empty string, and so on).

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.MaskField$Key) or value (org.apache.kafka.connect.transforms.MaskField$Value). +

+ + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
fieldsNames of fields to mask.listnon-empty listhigh
+

+
+
org.apache.kafka.connect.transforms.ValueToKey
+Replace the record key with a new key formed from a subset of fields in the record value. +

+ + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
fieldsField names on the record value to extract as the record key.listnon-empty listhigh
+

+
+
org.apache.kafka.connect.transforms.HoistField
+Wrap data using the specified field name in a Struct when schema present, or a Map in the case of schemaless data.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.HoistField$Key) or value (org.apache.kafka.connect.transforms.HoistField$Value). +

+ + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
fieldField name for the single field that will be created in the resulting Struct or Map.stringmedium
+

+
+
org.apache.kafka.connect.transforms.ExtractField
+Extract the specified field from a Struct when schema present, or a Map in the case of schemaless data. Any null values are passed through unmodified.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.ExtractField$Key) or value (org.apache.kafka.connect.transforms.ExtractField$Value). +

+ + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
fieldField name to extract.stringmedium
+

+
+
org.apache.kafka.connect.transforms.SetSchemaMetadata
+Set the schema name, version or both on the record's key (org.apache.kafka.connect.transforms.SetSchemaMetadata$Key) or value (org.apache.kafka.connect.transforms.SetSchemaMetadata$Value) schema. +

+ + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
schema.nameSchema name to set.stringnullhigh
schema.versionSchema version to set.intnullhigh
+

+
+
org.apache.kafka.connect.transforms.TimestampRouter
+Update the record's topic field as a function of the original topic value and the record timestamp.

This is mainly useful for sink connectors, since the topic field is often used to determine the equivalent entity name in the destination system(e.g. database table or search index name). +

+ + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
timestamp.formatFormat string for the timestamp that is compatible with java.text.SimpleDateFormat.stringyyyyMMddhigh
topic.formatFormat string which can contain ${topic} and ${timestamp} as placeholders for the topic and timestamp, respectively.string${topic}-${timestamp}high
+

+
+
org.apache.kafka.connect.transforms.RegexRouter
+Update the record topic using the configured regular expression and replacement string.

Under the hood, the regex is compiled to a java.util.regex.Pattern. If the pattern matches the input topic, java.util.regex.Matcher#replaceFirst() is used with the replacement string to obtain the new topic. +

+ + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
regexRegular expression to use for matching.stringvalid regexhigh
replacementReplacement string.stringhigh
+

+
+
org.apache.kafka.connect.transforms.Flatten
+Flatten a nested data structure, generating names for each field by concatenating the field names at each level with a configurable delimiter character. Applies to Struct when schema present, or a Map in the case of schemaless data. The default delimiter is '.'.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.Flatten$Key) or value (org.apache.kafka.connect.transforms.Flatten$Value). +

+ + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
delimiterDelimiter to insert between field names from the input record when generating field names for the output recordstring.medium
+

+
+
org.apache.kafka.connect.transforms.Cast
+Cast fields or the entire key or value to a specific type, e.g. to force an integer field to a smaller width. Only simple primitive types are supported -- integers, floats, boolean, and string.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.Cast$Key) or value (org.apache.kafka.connect.transforms.Cast$Value). +

+ + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
specList of fields and the type to cast them to of the form field1:type,field2:type to cast fields of Maps or Structs. A single type to cast the entire value. Valid types are int8, int16, int32, int64, float32, float64, boolean, and string.listlist of colon-delimited pairs, e.g. foo:bar,abc:xyzhigh
+

+
+
org.apache.kafka.connect.transforms.TimestampConverter
+Convert timestamps between different formats such as Unix epoch, strings, and Connect Date/Timestamp types.Applies to individual fields or to the entire value.

Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.TimestampConverter$Key) or value (org.apache.kafka.connect.transforms.TimestampConverter$Value). +

+ + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
target.typeThe desired timestamp representation: string, unix, Date, Time, or Timestampstringhigh
fieldThe field containing the timestamp, or empty if the entire value is a timestampstring""high
formatA SimpleDateFormat-compatible format for the timestamp. Used to generate the output when type=string or used to parse the input if the input is a string.string""medium
+

diff --git a/21/generated/consumer_config.html b/21/generated/consumer_config.html index e69de29..64b3059 100644 --- a/21/generated/consumer_config.html +++ b/21/generated/consumer_config.html @@ -0,0 +1,140 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + [...] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
key.deserializerDeserializer class for key that implements the org.apache.kafka.common.serialization.Deserializer interface.classhigh
value.deserializerDeserializer class for value that implements the org.apache.kafka.common.serialization.Deserializer interface.classhigh
bootstrap.serversA list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form host1:port1,host2:port2,.... Since these servers are just used for the initial connection to discover the full cluster me [...] +
fetch.min.bytesThe minimum amount of data the server should return for a fetch request. If insufficient data is available the request will wait for that much data to accumulate before answering the request. The default setting of 1 byte means that fetch requests are answered as soon as a single byte of data is available or the fetch request times out waiting for data to arrive. Setting this to something greater than 1 will cause the server to wait for larger amounts of [...] +
group.idA unique string that identifies the consumer group this consumer belongs to. This property is required if the consumer uses either the group management functionality by using subscribe(topic) or the Kafka-based offset management strategy.string""high
heartbeat.interval.msThe expected time between heartbeats to the consumer coordinator when using Kafka's group management facilities. Heartbeats are used to ensure that the consumer's session stays active and to facilitate rebalancing when new consumers join or leave the group. The value must be set lower than session.timeout.ms, but typically should be set no higher than 1/3 of that value. It can be adjusted even lower to control the expected time for norm [...] +
max.partition.fetch.bytesThe maximum amount of data per-partition the server will return. Records are fetched in batches by the consumer. If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress. The maximum record batch size accepted by the broker is defined via message.max.bytes (broker config) or max.message.bytes (topic c [...] +
session.timeout.msThe timeout used to detect consumer failures when using Kafka's group management facility. The consumer sends periodic heartbeats to indicate its liveness to the broker. If no heartbeats are received by the broker before the expiration of this session timeout, then the broker will remove this consumer from the group and initiate a rebalance. Note that the value must be in the allowable range as configured in the broker configuration by group.min. [...] +
ssl.key.passwordThe password of the private key in the key store file. This is optional for client.passwordnullhigh
ssl.keystore.locationThe location of the key store file. This is optional for client and can be used for two-way authentication for client.stringnullhigh
ssl.keystore.passwordThe store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured. passwordnullhigh
ssl.truststore.locationThe location of the trust store file. stringnullhigh
ssl.truststore.passwordThe password for the trust store file. If a password is not set access to the truststore is still available, but integrity checking is disabled.passwordnullhigh
auto.offset.resetWhat to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted):
  • earliest: automatically reset the offset to the earliest offset
  • latest: automatically reset the offset to the latest offset
  • none: throw exception to the consumer if no previous offset is found for the consumer's group
  • anything else: throw exception to the consumer.
    • +
client.dns.lookup

Controls how the client uses DNS lookups.

If set to use_all_dns_ips then, when the lookup returns multiple IP addresses for a hostname, they will all be attempted to connect to before failing the connection. Applies to both bootstrap and advertised servers.

If the value is resolve_canonical_bootstrap_servers_only each entry will be resolved and expanded into a list of canonical names.

string
connections.max.idle.msClose idle connections after the number of milliseconds specified by this config.long540000medium
default.api.timeout.msSpecifies the timeout (in milliseconds) for consumer APIs that could block. This configuration is used as the default timeout for all consumer operations that do not explicitly accept a timeout parameter.int60000[0,...]medium
enable.auto.commitIf true the consumer's offset will be periodically committed in the background.booleantruemedium
exclude.internal.topicsWhether records from internal topics (such as offsets) should be exposed to the consumer. If set to true the only way to receive records from an internal topic is subscribing to it.booleantruemedium
fetch.max.bytesThe maximum amount of data the server should return for a fetch request. Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum. The maximum record batch size accepted by the broker is defined via message.max.bytes (broker [...] +
isolation.level

Controls how to read messages written transactionally. If set to read_committed, consumer.poll() will only return transactional messages which have been committed. If set to read_uncommitted' (the default), consumer.poll() will return all messages, even transactional messages which have been aborted. Non-transactional messages will be returned unconditionally in either mode.

Messages will always be returned in offset or [...] +

max.poll.interval.msThe maximum delay between invocations of poll() when using consumer group management. This places an upper bound on the amount of time that the consumer can be idle before fetching more records. If poll() is not called before expiration of this timeout, then the consumer is considered failed and the group will rebalance in order to reassign the partitions to another member. int300000[1,...]mediu [...] +
max.poll.recordsThe maximum number of records returned in a single call to poll().int500[1,...]medium
partition.assignment.strategyThe class name of the partition assignment strategy that the client will use to distribute partition ownership amongst consumer instances when group management is usedlistclass org.apache.kafka.clients.consumer.RangeAssignornon-null stringmedium
receive.buffer.bytesThe size of the TCP receive buffer (SO_RCVBUF) to use when reading data. If the value is -1, the OS default will be used.int65536[-1,...]medium
request.timeout.msThe configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.int30000[0,...]medium
sasl.client.callback.handler.classThe fully qualified name of a SASL client callback handler class that implements the AuthenticateCallbackHandler interface.classnullmedium
sasl.jaas.configJAAS login context parameters for SASL connections in the format used by JAAS configuration files. JAAS configuration file format is described here. The format for the value is: 'loginModuleClass controlFlag (optionName=optionValue)*;'. For brokers, the config must be prefixed with listener prefix and SASL mechanism name in lower- [...] +
sasl.kerberos.service.nameThe Kerberos principal name that Kafka runs as. This can be defined either in Kafka's JAAS config or in Kafka's config.stringnullmedium
sasl.login.callback.handler.classThe fully qualified name of a SASL login callback handler class that implements the AuthenticateCallbackHandler interface. For brokers, login callback handler config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.callback.handler.class=com.example.CustomScramLoginCallbackHandlerclassnullmedium [...] +
sasl.login.classThe fully qualified name of a class that implements the Login interface. For brokers, login config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.class=com.example.CustomScramLoginclassnullmedium
sasl.mechanismSASL mechanism used for client connections. This may be any mechanism for which a security provider is available. GSSAPI is the default mechanism.stringGSSAPImedium
security.protocolProtocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.stringPLAINTEXTmedium
send.buffer.bytesThe size of the TCP send buffer (SO_SNDBUF) to use when sending data. If the value is -1, the OS default will be used.int131072[-1,...]medium
ssl.enabled.protocolsThe list of protocols enabled for SSL connections.listTLSv1.2,TLSv1.1,TLSv1medium
ssl.keystore.typeThe file format of the key store file. This is optional for client.stringJKSmedium
ssl.protocolThe SSL protocol used to generate the SSLContext. Default setting is TLS, which is fine for most cases. Allowed values in recent JVMs are TLS, TLSv1.1 and TLSv1.2. SSL, SSLv2 and SSLv3 may be supported in older JVMs, but their usage is discouraged due to known security vulnerabilities.stringTLSmedium
ssl.providerThe name of the security provider used for SSL connections. Default value is the default security provider of the JVM.stringnullmedium
ssl.truststore.typeThe file format of the trust store file.stringJKSmedium
auto.commit.interval.msThe frequency in milliseconds that the consumer offsets are auto-committed to Kafka if enable.auto.commit is set to true.int5000[0,...]low
check.crcsAutomatically check the CRC32 of the records consumed. This ensures no on-the-wire or on-disk corruption to the messages occurred. This check adds some overhead, so it may be disabled in cases seeking extreme performance.booleantruelow
client.idAn id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.string""low
fetch.max.wait.msThe maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.int500[0,...]low
interceptor.classesA list of classes to use as interceptors. Implementing the org.apache.kafka.clients.consumer.ConsumerInterceptor interface allows you to intercept (and possibly mutate) records received by the consumer. By default, there are no interceptors.list""non-null stringlow
metadata.max.age.msThe period of time in milliseconds after which we force a refresh of metadata even if we haven't seen any partition leadership changes to proactively discover any new brokers or partitions.long300000[0,...]low
metric.reportersA list of classes to use as metrics reporters. Implementing the org.apache.kafka.common.metrics.MetricsReporter interface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics.list""non-null stringlow
metrics.num.samplesThe number of samples maintained to compute metrics.int2[1,...]low
metrics.recording.levelThe highest recording level for metrics.stringINFO[INFO, DEBUG]low
metrics.sample.window.msThe window of time a metrics sample is computed over.long30000[0,...]low
reconnect.backoff.max.msThe maximum amount of time in milliseconds to wait when reconnecting to a broker that has repeatedly failed to connect. If provided, the backoff per host will increase exponentially for each consecutive connection failure, up to this maximum. After calculating the backoff increase, 20% random jitter is added to avoid connection storms.long1000[0,...]low
reconnect.backoff.msThe base amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop. This backoff applies to all connection attempts by the client to a broker.long50[0,...]low
retry.backoff.msThe amount of time to wait before attempting to retry a failed request to a given topic partition. This avoids repeatedly sending requests in a tight loop under some failure scenarios.long100[0,...]low
sasl.kerberos.kinit.cmdKerberos kinit command path.string/usr/bin/kinitlow
sasl.kerberos.min.time.before.reloginLogin thread sleep time between refresh attempts.long60000low
sasl.kerberos.ticket.renew.jitterPercentage of random jitter added to the renewal time.double0.05low
sasl.kerberos.ticket.renew.window.factorLogin thread will sleep until the specified window factor of time from last refresh to ticket's expiry has been reached, at which time it will try to renew the ticket.double0.8low
sasl.login.refresh.buffer.secondsThe amount of buffer time before credential expiration to maintain when refreshing a credential, in seconds. If a refresh would otherwise occur closer to expiration than the number of buffer seconds then the refresh will be moved up to maintain as much of the buffer time as possible. Legal values are between 0 and 3600 (1 hour); a default value of 300 (5 minutes) is used if no value is specified. This value and sasl.login.refresh.min.pe [...] +
sasl.login.refresh.min.period.secondsThe desired minimum time for the login refresh thread to wait before refreshing a credential, in seconds. Legal values are between 0 and 900 (15 minutes); a default value of 60 (1 minute) is used if no value is specified. This value and sasl.login.refresh.buffer.seconds are both ignored if their sum exceeds the remaining lifetime of a credential. Currently applies only to OAUTHBEARER.short60 +sasl.login.refresh.window.factorLogin refresh thread will sleep until the specified window factor relative to the credential's lifetime has been reached, at which time it will try to refresh the credential. Legal values are between 0.5 (50%) and 1.0 (100%) inclusive; a default value of 0.8 (80%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.8[0.5,...,1.0]low
sasl.login.refresh.window.jitterThe maximum amount of random jitter relative to the credential's lifetime that is added to the login refresh thread's sleep time. Legal values are between 0 and 0.25 (25%) inclusive; a default value of 0.05 (5%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.05[0.0,...,0.25]low
ssl.cipher.suitesA list of cipher suites. This is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. By default all the available cipher suites are supported.listnulllow
ssl.endpoint.identification.algorithmThe endpoint identification algorithm to validate server hostname using server certificate. stringhttpslow
ssl.keymanager.algorithmThe algorithm used by key manager factory for SSL connections. Default value is the key manager factory algorithm configured for the Java Virtual Machine.stringSunX509low
ssl.secure.random.implementationThe SecureRandom PRNG implementation to use for SSL cryptography operations. stringnulllow
ssl.trustmanager.algorithmThe algorithm used by trust manager factory for SSL connections. Default value is the trust manager factory algorithm configured for the Java Virtual Machine.stringPKIXlow
diff --git a/21/generated/consumer_metrics.html b/21/generated/consumer_metrics.html index e69de29..7844d67 100644 --- a/21/generated/consumer_metrics.html +++ b/21/generated/consumer_metrics.html @@ -0,0 +1,79 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{client-id}"
Attribute nameDescription
bytes-consumed-rateThe average number of bytes consumed per second
bytes-consumed-totalThe total number of bytes consumed
fetch-latency-avgThe average time taken for a fetch request.
fetch-latency-maxThe max time taken for any fetch request.
fetch-rateThe number of fetch requests per second.
fetch-size-avgThe average number of bytes fetched per request
fetch-size-maxThe maximum number of bytes fetched per request
fetch-throttle-time-avgThe average throttle time in ms
fetch-throttle-time-maxThe maximum throttle time in ms
fetch-totalThe total number of fetch requests.
records-consumed-rateThe average number of records consumed per second
records-consumed-totalThe total number of records consumed
records-lag-maxThe maximum lag in terms of number of records for any partition in this window
records-lead-minThe minimum lead in terms of number of records for any partition in this window
records-per-request-avgThe average number of records in each request
kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{client-id}",topic="{topic}"
Attribute nameDescription
bytes-consumed-rateThe average number of bytes consumed per second for a topic
bytes-consumed-totalThe total number of bytes consumed for a topic
fetch-size-avgThe average number of bytes fetched per request for a topic
fetch-size-maxThe maximum number of bytes fetched per request for a topic
records-consumed-rateThe average number of records consumed per second for a topic
records-consumed-totalThe total number of records consumed for a topic
records-per-request-avgThe average number of records in each request for a topic
kafka.consumer:type=consumer-fetch-manager-metrics,partition="{partition}",topic="{topic}",client-id="{client-id}"
Attribute nameDescription
records-lagThe latest lag of the partition
records-lag-avgThe average lag of the partition
records-lag-maxThe max lag of the partition
records-leadThe latest lead of the partition
records-lead-avgThe average lead of the partition
records-lead-minThe min lead of the partition
diff --git a/21/generated/kafka_config.html b/21/generated/kafka_config.html index e69de29..8be1583 100644 --- a/21/generated/kafka_config.html +++ b/21/generated/kafka_config.html @@ -0,0 +1,392 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +< [...] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + [...] + + + + + + + + + + [...] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportanceDynamic Update Mode
zookeeper.connectSpecifies the ZooKeeper connection string in the form hostname:port where host and port are the host and port of a ZooKeeper server. To allow connecting through other ZooKeeper nodes when that ZooKeeper machine is down you can also specify multiple hosts in the form hostname1:port1,hostname2:port2,hostname3:port3. +The server can also have a ZooKeeper chroot path as part of its ZooKeeper connection string which puts its data under some path in the global ZooKeeper namespace. For example to give a chroot path of /chroot/path you would give the connection string as hostname1:port1,hostname2:port2,hostname3:port3/chroot/path.stringhighread-only
advertised.host.nameDEPRECATED: only used when advertised.listeners or listeners are not set. Use advertised.listeners instead. +Hostname to publish to ZooKeeper for clients to use. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, it will use the value for host.name if configured. Otherwise it will use the value returned from java.net.InetAddress.getCanonicalHostName().stringnullhighread-only
advertised.listenersListeners to publish to ZooKeeper for clients to use, if different than the listeners config property. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, the value for listeners will be used. Unlike listeners it is not valid to advertise the 0.0.0.0 meta-address.stringnullhighp [...] +
advertised.portDEPRECATED: only used when advertised.listeners or listeners are not set. Use advertised.listeners instead. +The port to publish to ZooKeeper for clients to use. In IaaS environments, this may need to be different from the port to which the broker binds. If this is not set, it will publish the same port that the broker binds to.intnullhighread-only
auto.create.topics.enableEnable auto creation of topic on the serverbooleantruehighread-only
auto.leader.rebalance.enableEnables auto leader balancing. A background thread checks and triggers leader balance if required at regular intervalsbooleantruehighread-only
background.threadsThe number of threads to use for various background processing tasksint10[1,...]highcluster-wide
broker.idThe broker id for this server. If unset, a unique broker id will be generated.To avoid conflicts between zookeeper generated broker id's and user configured broker id's, generated broker ids start from reserved.broker.max.id + 1.int-1highread-only
compression.typeSpecify the final compression type for a given topic. This configuration accepts the standard compression codecs ('gzip', 'snappy', 'lz4', 'zstd'). It additionally accepts 'uncompressed' which is equivalent to no compression; and 'producer' which means retain the original compression codec set by the producer.stringproducerhighcluster-wide
delete.topic.enableEnables delete topic. Delete topic through the admin tool will have no effect if this config is turned offbooleantruehighread-only
host.nameDEPRECATED: only used when listeners is not set. Use listeners instead. +hostname of broker. If this is set, it will only bind to this address. If this is not set, it will bind to all interfacesstring""highread-only
leader.imbalance.check.interval.secondsThe frequency with which the partition rebalance check is triggered by the controllerlong300highread-only
leader.imbalance.per.broker.percentageThe ratio of leader imbalance allowed per broker. The controller would trigger a leader balance if it goes above this value per broker. The value is specified in percentage.int10highread-only
listenersListener List - Comma-separated list of URIs we will listen on and the listener names. If the listener name is not a security protocol, listener.security.protocol.map must also be set. + Specify hostname as 0.0.0.0 to bind to all interfaces. + Leave hostname empty to bind to default interface. + Examples of legal listener lists: + PLAINTEXT://myhost:9092,SSL://:9091 + CLIENT://0.0.0.0:9092,REPLICATION://localhost:9093 +stringnullhighper-broker
log.dirThe directory in which the log data is kept (supplemental for log.dirs property)string/tmp/kafka-logshighread-only
log.dirsThe directories in which the log data is kept. If not set, the value in log.dir is usedstringnullhighread-only
log.flush.interval.messagesThe number of messages accumulated on a log partition before messages are flushed to disk long9223372036854775807[1,...]highcluster-wide
log.flush.interval.msThe maximum time in ms that a message in any topic is kept in memory before flushed to disk. If not set, the value in log.flush.scheduler.interval.ms is usedlongnullhighcluster-wide
log.flush.offset.checkpoint.interval.msThe frequency with which we update the persistent record of the last flush which acts as the log recovery pointint60000[0,...]highread-only
log.flush.scheduler.interval.msThe frequency in ms that the log flusher checks whether any log needs to be flushed to disklong9223372036854775807highread-only
log.flush.start.offset.checkpoint.interval.msThe frequency with which we update the persistent record of log start offsetint60000[0,...]highread-only
log.retention.bytesThe maximum size of the log before deleting itlong-1highcluster-wide
log.retention.hoursThe number of hours to keep a log file before deleting it (in hours), tertiary to log.retention.ms propertyint168highread-only
log.retention.minutesThe number of minutes to keep a log file before deleting it (in minutes), secondary to log.retention.ms property. If not set, the value in log.retention.hours is usedintnullhighread-only
log.retention.msThe number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is usedlongnullhighcluster-wide
log.roll.hoursThe maximum time before a new log segment is rolled out (in hours), secondary to log.roll.ms propertyint168[1,...]highread-only
log.roll.jitter.hoursThe maximum jitter to subtract from logRollTimeMillis (in hours), secondary to log.roll.jitter.ms propertyint0[0,...]highread-only
log.roll.jitter.msThe maximum jitter to subtract from logRollTimeMillis (in milliseconds). If not set, the value in log.roll.jitter.hours is usedlongnullhighcluster-wide
log.roll.msThe maximum time before a new log segment is rolled out (in milliseconds). If not set, the value in log.roll.hours is usedlongnullhighcluster-wide
log.segment.bytesThe maximum size of a single log fileint1073741824[14,...]highcluster-wide
log.segment.delete.delay.msThe amount of time to wait before deleting a file from the filesystemlong60000[0,...]highcluster-wide
message.max.bytes

The largest record batch size allowed by Kafka. If this is increased and there are consumers older than 0.10.2, the consumers' fetch size must also be increased so that the they can fetch record batches this large.

In the latest message format version, records are always grouped into batches for efficiency. In previous message format versions, uncompressed records are not grouped into batches and this limit only applies to a single record in th [...] +

min.insync.replicasWhen a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend).
When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical scenario would be to create [...] +
num.io.threadsThe number of threads that the server uses for processing requests, which may include disk I/Oint8[1,...]highcluster-wide
num.network.threadsThe number of threads that the server uses for receiving requests from the network and sending responses to the networkint3[1,...]highcluster-wide
num.recovery.threads.per.data.dirThe number of threads per data directory to be used for log recovery at startup and flushing at shutdownint1[1,...]highcluster-wide
num.replica.alter.log.dirs.threadsThe number of threads that can move replicas between log directories, which may include disk I/Ointnullhighread-only
num.replica.fetchersNumber of fetcher threads used to replicate messages from a source broker. Increasing this value can increase the degree of I/O parallelism in the follower broker.int1highcluster-wide
offset.metadata.max.bytesThe maximum size for a metadata entry associated with an offset commitint4096highread-only
offsets.commit.required.acksThe required acks before the commit can be accepted. In general, the default (-1) should not be overriddenshort-1highread-only
offsets.commit.timeout.msOffset commit will be delayed until all replicas for the offsets topic receive the commit or this timeout is reached. This is similar to the producer request timeout.int5000[1,...]highread-only
offsets.load.buffer.sizeBatch size for reading from the offsets segments when loading offsets into the cache (soft-limit, overridden if records are too large).int5242880[1,...]highread-only
offsets.retention.check.interval.msFrequency at which to check for stale offsetslong600000[1,...]highread-only
offsets.retention.minutesAfter a consumer group loses all its consumers (i.e. becomes empty) its offsets will be kept for this retention period before getting discarded. For standalone consumers (using manual assignment), offsets will be expired after the time of last commit plus this retention period.int10080[1,...]highread-only
offsets.topic.compression.codecCompression codec for the offsets topic - compression may be used to achieve "atomic" commitsint0highread-only
offsets.topic.num.partitionsThe number of partitions for the offset commit topic (should not change after deployment)int50[1,...]highread-only
offsets.topic.replication.factorThe replication factor for the offsets topic (set higher to ensure availability). Internal topic creation will fail until the cluster size meets this replication factor requirement.short3[1,...]highread-only
offsets.topic.segment.bytesThe offsets topic segment bytes should be kept relatively small in order to facilitate faster log compaction and cache loadsint104857600[1,...]highread-only
portDEPRECATED: only used when listeners is not set. Use listeners instead. +the port to listen and accept connections onint9092highread-only
queued.max.requestsThe number of queued requests allowed before blocking the network threadsint500[1,...]highread-only
quota.consumer.defaultDEPRECATED: Used only when dynamic default quotas are not configured for or in Zookeeper. Any consumer distinguished by clientId/consumer group will get throttled if it fetches more bytes than this value per-secondlong9223372036854775807[1,...]highread-only
quota.producer.defaultDEPRECATED: Used only when dynamic default quotas are not configured for , or in Zookeeper. Any producer distinguished by clientId will get throttled if it produces more bytes than this value per-secondlong9223372036854775807[1,...]highread-only
replica.fetch.min.bytesMinimum bytes expected for each fetch response. If not enough bytes, wait up to replicaMaxWaitTimeMsint1highread-only
replica.fetch.wait.max.msmax wait time for each fetcher request issued by follower replicas. This value should always be less than the replica.lag.time.max.ms at all times to prevent frequent shrinking of ISR for low throughput topicsint500highread-only
replica.high.watermark.checkpoint.interval.msThe frequency with which the high watermark is saved out to disklong5000highread-only
replica.lag.time.max.msIf a follower hasn't sent any fetch requests or hasn't consumed up to the leaders log end offset for at least this time, the leader will remove the follower from isrlong10000highread-only
replica.socket.receive.buffer.bytesThe socket receive buffer for network requestsint65536highread-only
replica.socket.timeout.msThe socket timeout for network requests. Its value should be at least replica.fetch.wait.max.msint30000highread-only
request.timeout.msThe configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.int30000highread-only
socket.receive.buffer.bytesThe SO_RCVBUF buffer of the socket sever sockets. If the value is -1, the OS default will be used.int102400highread-only
socket.request.max.bytesThe maximum number of bytes in a socket requestint104857600[1,...]highread-only
socket.send.buffer.bytesThe SO_SNDBUF buffer of the socket sever sockets. If the value is -1, the OS default will be used.int102400highread-only
transaction.max.timeout.msThe maximum allowed timeout for transactions. If a client’s requested transaction time exceed this, then the broker will return an error in InitProducerIdRequest. This prevents a client from too large of a timeout, which can stall consumers reading from topics included in the transaction.int900000[1,...]highread-only
transaction.state.log.load.buffer.sizeBatch size for reading from the transaction log segments when loading producer ids and transactions into the cache (soft-limit, overridden if records are too large).int5242880[1,...]highread-only
transaction.state.log.min.isrOverridden min.insync.replicas config for the transaction topic.int2[1,...]highread-only
transaction.state.log.num.partitionsThe number of partitions for the transaction topic (should not change after deployment).int50[1,...]highread-only
transaction.state.log.replication.factorThe replication factor for the transaction topic (set higher to ensure availability). Internal topic creation will fail until the cluster size meets this replication factor requirement.short3[1,...]highread-only
transaction.state.log.segment.bytesThe transaction topic segment bytes should be kept relatively small in order to facilitate faster log compaction and cache loadsint104857600[1,...]highread-only
transactional.id.expiration.msThe maximum amount of time in ms that the transaction coordinator will wait before proactively expire a producer's transactional id without receiving any transaction status updates from it.int604800000[1,...]highread-only
unclean.leader.election.enableIndicates whether to enable replicas not in the ISR set to be elected as leader as a last resort, even though doing so may result in data lossbooleanfalsehighcluster-wide
zookeeper.connection.timeout.msThe max time that the client waits to establish a connection to zookeeper. If not set, the value in zookeeper.session.timeout.ms is usedintnullhighread-only
zookeeper.max.in.flight.requestsThe maximum number of unacknowledged requests the client will send to Zookeeper before blocking.int10[1,...]highread-only
zookeeper.session.timeout.msZookeeper session timeoutint6000highread-only
zookeeper.set.aclSet client to use secure ACLsbooleanfalsehighread-only
broker.id.generation.enableEnable automatic broker id generation on the server. When enabled the value configured for reserved.broker.max.id should be reviewed.booleantruemediumread-only
broker.rackRack of the broker. This will be used in rack aware replication assignment for fault tolerance. Examples: `RACK1`, `us-east-1d`stringnullmediumread-only
connections.max.idle.msIdle connections timeout: the server socket processor threads close the connections that idle more than thislong600000mediumread-only
controlled.shutdown.enableEnable controlled shutdown of the serverbooleantruemediumread-only
controlled.shutdown.max.retriesControlled shutdown can fail for multiple reasons. This determines the number of retries when such failure happensint3mediumread-only
controlled.shutdown.retry.backoff.msBefore each retry, the system needs time to recover from the state that caused the previous failure (Controller fail over, replica lag etc). This config determines the amount of time to wait before retrying.long5000mediumread-only
controller.socket.timeout.msThe socket timeout for controller-to-broker channelsint30000mediumread-only
default.replication.factordefault replication factors for automatically created topicsint1mediumread-only
delegation.token.expiry.time.msThe token validity time in miliseconds before the token needs to be renewed. Default value 1 day.long86400000[1,...]mediumread-only
delegation.token.master.keyMaster/secret key to generate and verify delegation tokens. Same key must be configured across all the brokers. If the key is not set or set to empty string, brokers will disable the delegation token support.passwordnullmediumread-only
delegation.token.max.lifetime.msThe token has a maximum lifetime beyond which it cannot be renewed anymore. Default value 7 days.long604800000[1,...]mediumread-only
delete.records.purgatory.purge.interval.requestsThe purge interval (in number of requests) of the delete records request purgatoryint1mediumread-only
fetch.purgatory.purge.interval.requestsThe purge interval (in number of requests) of the fetch request purgatoryint1000mediumread-only
group.initial.rebalance.delay.msThe amount of time the group coordinator will wait for more consumers to join a new group before performing the first rebalance. A longer delay means potentially fewer rebalances, but increases the time until processing begins.int3000mediumread-only
group.max.session.timeout.msThe maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.int300000mediumread-only
group.min.session.timeout.msThe minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating, which can overwhelm broker resources.int6000mediumread-only
inter.broker.listener.nameName of listener used for communication between brokers. If this is unset, the listener name is defined by security.inter.broker.protocol. It is an error to set this and security.inter.broker.protocol properties at the same time.stringnullmediumread-only
inter.broker.protocol.versionSpecify which version of the inter-broker protocol will be used. + This is typically bumped after all brokers were upgraded to a new version. + Example of some valid values are: 0.8.0, 0.8.1, 0.8.1.1, 0.8.2, 0.8.2.0, 0.8.2.1, 0.9.0.0, 0.9.0.1 Check ApiVersion for the full list.string2.1-IV2kafka.api.ApiVersionValidator$@4eb7f003mediumread-only
log.cleaner.backoff.msThe amount of time to sleep when there are no logs to cleanlong15000[0,...]mediumcluster-wide
log.cleaner.dedupe.buffer.sizeThe total memory used for log deduplication across all cleaner threadslong134217728mediumcluster-wide
log.cleaner.delete.retention.msHow long are delete records retained?long86400000mediumcluster-wide
log.cleaner.enableEnable the log cleaner process to run on the server. Should be enabled if using any topics with a cleanup.policy=compact including the internal offsets topic. If disabled those topics will not be compacted and continually grow in size.booleantruemediumread-only
log.cleaner.io.buffer.load.factorLog cleaner dedupe buffer load factor. The percentage full the dedupe buffer can become. A higher value will allow more log to be cleaned at once but will lead to more hash collisionsdouble0.9mediumcluster-wide
log.cleaner.io.buffer.sizeThe total memory used for log cleaner I/O buffers across all cleaner threadsint524288[0,...]mediumcluster-wide
log.cleaner.io.max.bytes.per.secondThe log cleaner will be throttled so that the sum of its read and write i/o will be less than this value on averagedouble1.7976931348623157E308mediumcluster-wide
log.cleaner.min.cleanable.ratioThe minimum ratio of dirty log to total log for a log to eligible for cleaningdouble0.5mediumcluster-wide
log.cleaner.min.compaction.lag.msThe minimum time a message will remain uncompacted in the log. Only applicable for logs that are being compacted.long0mediumcluster-wide
log.cleaner.threadsThe number of background threads to use for log cleaningint1[0,...]mediumcluster-wide
log.cleanup.policyThe default cleanup policy for segments beyond the retention window. A comma separated list of valid policies. Valid policies are: "delete" and "compact"listdelete[compact, delete]mediumcluster-wide
log.index.interval.bytesThe interval with which we add an entry to the offset indexint4096[0,...]mediumcluster-wide
log.index.size.max.bytesThe maximum size in bytes of the offset indexint10485760[4,...]mediumcluster-wide
log.message.format.versionSpecify the message format version the broker will use to append messages to the logs. The value should be a valid ApiVersion. Some examples are: 0.8.2, 0.9.0.0, 0.10.0, check ApiVersion for more details. By setting a particular message format version, the user is certifying that all the existing messages on disk are smaller or equal than the specified version. Setting this value incorrectly will cause consumers with older versions to break as [...] +
log.message.timestamp.difference.max.msThe maximum difference allowed between the timestamp when a broker receives a message and the timestamp specified in the message. If log.message.timestamp.type=CreateTime, a message will be rejected if the difference in timestamp exceeds this threshold. This configuration is ignored if log.message.timestamp.type=LogAppendTime.The maximum timestamp difference allowed should be no greater than log.retention.ms to avoid unnecessarily [...] +
log.message.timestamp.typeDefine whether the timestamp in the message is message create time or log append time. The value should be either `CreateTime` or `LogAppendTime`stringCreateTime[CreateTime, LogAppendTime]mediumcluster-wide
log.preallocateShould pre allocate file when create new segment? If you are using Kafka on Windows, you probably need to set it to true.booleanfalsemediumcluster-wide
log.retention.check.interval.msThe frequency in milliseconds that the log cleaner checks whether any log is eligible for deletionlong300000[1,...]mediumread-only
max.connections.per.ipThe maximum number of connections we allow from each ip address. This can be set to 0 if there are overrides configured using max.connections.per.ip.overrides propertyint2147483647[0,...]mediumcluster-wide
max.connections.per.ip.overridesA comma-separated list of per-ip or hostname overrides to the default maximum number of connections. An example value is "hostName:100,127.0.0.1:200"string""mediumcluster-wide
max.incremental.fetch.session.cache.slotsThe maximum number of incremental fetch sessions that we will maintain.int1000[0,...]mediumread-only
num.partitionsThe default number of log partitions per topicint1[1,...]mediumread-only
password.encoder.old.secretThe old secret that was used for encoding dynamically configured passwords. This is required only when the secret is updated. If specified, all dynamically encoded passwords are decoded using this old secret and re-encoded using password.encoder.secret when broker starts up.passwordnullmediumread-only
password.encoder.secretThe secret used for encoding dynamically configured passwords for this broker.passwordnullmediumread-only
principal.builder.classThe fully qualified name of a class that implements the KafkaPrincipalBuilder interface, which is used to build the KafkaPrincipal object used during authorization. This config also supports the deprecated PrincipalBuilder interface which was previously used for client authentication over SSL. If no principal builder is defined, the default behavior depends on the security protocol in use. For SSL authentication, the principal name will be the dis [...] +
producer.purgatory.purge.interval.requestsThe purge interval (in number of requests) of the producer request purgatoryint1000mediumread-only
queued.max.request.bytesThe number of queued bytes allowed before no more requests are readlong-1mediumread-only
replica.fetch.backoff.msThe amount of time to sleep when fetch partition error occurs.int1000[0,...]mediumread-only
replica.fetch.max.bytesThe number of bytes of messages to attempt to fetch for each partition. This is not an absolute maximum, if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made. The maximum record batch size accepted by the broker is defined via message.max.bytes (broker config) or max.message.bytes (topic config).
replica.fetch.response.max.bytesMaximum bytes expected for the entire fetch response. Records are fetched in batches, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made. As such, this is not an absolute maximum. The maximum record batch size accepted by the broker is defined via message.max.bytes (broker config) or max.messag [...] +
reserved.broker.max.idMax number that can be used for a broker.idint1000[0,...]mediumread-only
sasl.client.callback.handler.classThe fully qualified name of a SASL client callback handler class that implements the AuthenticateCallbackHandler interface.classnullmediumread-only
sasl.enabled.mechanismsThe list of SASL mechanisms enabled in the Kafka server. The list may contain any mechanism for which a security provider is available. Only GSSAPI is enabled by default.listGSSAPImediumper-broker
sasl.jaas.configJAAS login context parameters for SASL connections in the format used by JAAS configuration files. JAAS configuration file format is described here. The format for the value is: 'loginModuleClass controlFlag (optionName=optionValue)*;'. For brokers, the config must be prefixed with listener prefix and SASL mechanism name in lower- [...] +
sasl.kerberos.kinit.cmdKerberos kinit command path.string/usr/bin/kinitmediumper-broker
sasl.kerberos.min.time.before.reloginLogin thread sleep time between refresh attempts.long60000mediumper-broker
sasl.kerberos.principal.to.local.rulesA list of rules for mapping from principal names to short names (typically operating system usernames). The rules are evaluated in order and the first rule that matches a principal name is used to map it to a short name. Any later rules in the list are ignored. By default, principal names of the form {username}/{hostname}@{REALM} are mapped to {username}. For more details on the format please see security [...] +
sasl.kerberos.service.nameThe Kerberos principal name that Kafka runs as. This can be defined either in Kafka's JAAS config or in Kafka's config.stringnullmediumper-broker
sasl.kerberos.ticket.renew.jitterPercentage of random jitter added to the renewal time.double0.05mediumper-broker
sasl.kerberos.ticket.renew.window.factorLogin thread will sleep until the specified window factor of time from last refresh to ticket's expiry has been reached, at which time it will try to renew the ticket.double0.8mediumper-broker
sasl.login.callback.handler.classThe fully qualified name of a SASL login callback handler class that implements the AuthenticateCallbackHandler interface. For brokers, login callback handler config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.callback.handler.class=com.example.CustomScramLoginCallbackHandlerclassnullmedium [...] +
sasl.login.classThe fully qualified name of a class that implements the Login interface. For brokers, login config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.class=com.example.CustomScramLoginclassnullmediumread-only
sasl.login.refresh.buffer.secondsThe amount of buffer time before credential expiration to maintain when refreshing a credential, in seconds. If a refresh would otherwise occur closer to expiration than the number of buffer seconds then the refresh will be moved up to maintain as much of the buffer time as possible. Legal values are between 0 and 3600 (1 hour); a default value of 300 (5 minutes) is used if no value is specified. This value and sasl.login.refresh.min.pe [...] +
sasl.login.refresh.min.period.secondsThe desired minimum time for the login refresh thread to wait before refreshing a credential, in seconds. Legal values are between 0 and 900 (15 minutes); a default value of 60 (1 minute) is used if no value is specified. This value and sasl.login.refresh.buffer.seconds are both ignored if their sum exceeds the remaining lifetime of a credential. Currently applies only to OAUTHBEARER.short60 +sasl.login.refresh.window.factorLogin refresh thread will sleep until the specified window factor relative to the credential's lifetime has been reached, at which time it will try to refresh the credential. Legal values are between 0.5 (50%) and 1.0 (100%) inclusive; a default value of 0.8 (80%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.8mediumper-broker
sasl.login.refresh.window.jitterThe maximum amount of random jitter relative to the credential's lifetime that is added to the login refresh thread's sleep time. Legal values are between 0 and 0.25 (25%) inclusive; a default value of 0.05 (5%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.05mediumper-broker
sasl.mechanism.inter.broker.protocolSASL mechanism used for inter-broker communication. Default is GSSAPI.stringGSSAPImediumper-broker
sasl.server.callback.handler.classThe fully qualified name of a SASL server callback handler class that implements the AuthenticateCallbackHandler interface. Server callback handlers must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.plain.sasl.server.callback.handler.class=com.example.CustomPlainCallbackHandler.classnullmediumread-only
security.inter.broker.protocolSecurity protocol used to communicate between brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. It is an error to set this and inter.broker.listener.name properties at the same time.stringPLAINTEXTmediumread-only
ssl.cipher.suitesA list of cipher suites. This is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. By default all the available cipher suites are supported.list""mediumper-broker
ssl.client.authConfigures kafka broker to request client authentication. The following settings are common:
  • ssl.client.auth=required If set to required client authentication is required.
  • ssl.client.auth=requested This means client authentication is optional. unlike requested , if this option is set client can choose not to provide authentication information about itself
  • ssl.client.auth=none This means client authe [...] +
ssl.enabled.protocolsThe list of protocols enabled for SSL connections.listTLSv1.2,TLSv1.1,TLSv1mediumper-broker
ssl.key.passwordThe password of the private key in the key store file. This is optional for client.passwordnullmediumper-broker
ssl.keymanager.algorithmThe algorithm used by key manager factory for SSL connections. Default value is the key manager factory algorithm configured for the Java Virtual Machine.stringSunX509mediumper-broker
ssl.keystore.locationThe location of the key store file. This is optional for client and can be used for two-way authentication for client.stringnullmediumper-broker
ssl.keystore.passwordThe store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured. passwordnullmediumper-broker
ssl.keystore.typeThe file format of the key store file. This is optional for client.stringJKSmediumper-broker
ssl.protocolThe SSL protocol used to generate the SSLContext. Default setting is TLS, which is fine for most cases. Allowed values in recent JVMs are TLS, TLSv1.1 and TLSv1.2. SSL, SSLv2 and SSLv3 may be supported in older JVMs, but their usage is discouraged due to known security vulnerabilities.stringTLSmediumper-broker
ssl.providerThe name of the security provider used for SSL connections. Default value is the default security provider of the JVM.stringnullmediumper-broker
ssl.trustmanager.algorithmThe algorithm used by trust manager factory for SSL connections. Default value is the trust manager factory algorithm configured for the Java Virtual Machine.stringPKIXmediumper-broker
ssl.truststore.locationThe location of the trust store file. stringnullmediumper-broker
ssl.truststore.passwordThe password for the trust store file. If a password is not set access to the truststore is still available, but integrity checking is disabled.passwordnullmediumper-broker
ssl.truststore.typeThe file format of the trust store file.stringJKSmediumper-broker
alter.config.policy.class.nameThe alter configs policy class that should be used for validation. The class should implement the org.apache.kafka.server.policy.AlterConfigPolicy interface.classnulllowread-only
alter.log.dirs.replication.quota.window.numThe number of samples to retain in memory for alter log dirs replication quotasint11[1,...]lowread-only
alter.log.dirs.replication.quota.window.size.secondsThe time span of each sample for alter log dirs replication quotasint1[1,...]lowread-only
authorizer.class.nameThe authorizer class that should be used for authorizationstring""lowread-only
client.quota.callback.classThe fully qualified name of a class that implements the ClientQuotaCallback interface, which is used to determine quota limits applied to client requests. By default, , or quotas stored in ZooKeeper are applied. For any given request, the most specific quota that matches the user principal of the session and the client-id of the request is applied.classnull
connection.failed.authentication.delay.msConnection close delay on failed authentication: this is the time (in milliseconds) by which connection close will be delayed on authentication failure. This must be configured to be less than connections.max.idle.ms to prevent connection timeout.int100[0,...]lowread-only
create.topic.policy.class.nameThe create topic policy class that should be used for validation. The class should implement the org.apache.kafka.server.policy.CreateTopicPolicy interface.classnulllowread-only
delegation.token.expiry.check.interval.msScan interval to remove expired delegation tokens.long3600000[1,...]lowread-only
kafka.metrics.polling.interval.secsThe metrics polling interval (in seconds) which can be used in kafka.metrics.reporters implementations.int10[1,...]lowread-only
kafka.metrics.reportersA list of classes to use as Yammer metrics custom reporters. The reporters should implement kafka.metrics.KafkaMetricsReporter trait. If a client wants to expose JMX operations on a custom reporter, the custom reporter needs to additionally implement an MBean trait that extends kafka.metrics.KafkaMetricsReporterMBean trait so that the registered MBean is compliant with the standard MBean convention.list
listener.security.protocol.mapMap between listener names and security protocols. This must be defined for the same security protocol to be usable in more than one port or IP. For example, internal and external traffic can be separated even if SSL is required for both. Concretely, the user could define listeners with names INTERNAL and EXTERNAL and this property as: `INTERNAL:SSL,EXTERNAL:SSL`. As shown, key and value are separated by a colon and map entries are separate [...] +
log.message.downconversion.enableThis configuration controls whether down-conversion of message formats is enabled to satisfy consume requests. When set to false, broker will not perform down-conversion for consumers expecting an older message format. The broker responds with UNSUPPORTED_VERSION error for consume requests from such older clients. This configurationdoes not apply to any message format conversion that might be required for replic [...] +
metric.reportersA list of classes to use as metrics reporters. Implementing the org.apache.kafka.common.metrics.MetricsReporter interface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics.list""lowcluster-wide
metrics.num.samplesThe number of samples maintained to compute metrics.int2[1,...]lowread-only
metrics.recording.levelThe highest recording level for metrics.stringINFOlowread-only
metrics.sample.window.msThe window of time a metrics sample is computed over.long30000[1,...]lowread-only
password.encoder.cipher.algorithmThe Cipher algorithm used for encoding dynamically configured passwords.stringAES/CBC/PKCS5Paddinglowread-only
password.encoder.iterationsThe iteration count used for encoding dynamically configured passwords.int4096[1024,...]lowread-only
password.encoder.key.lengthThe key length used for encoding dynamically configured passwords.int128[8,...]lowread-only
password.encoder.keyfactory.algorithmThe SecretKeyFactory algorithm used for encoding dynamically configured passwords. Default is PBKDF2WithHmacSHA512 if available and PBKDF2WithHmacSHA1 otherwise.stringnulllowread-only
quota.window.numThe number of samples to retain in memory for client quotasint11[1,...]lowread-only
quota.window.size.secondsThe time span of each sample for client quotasint1[1,...]lowread-only
replication.quota.window.numThe number of samples to retain in memory for replication quotasint11[1,...]lowread-only
replication.quota.window.size.secondsThe time span of each sample for replication quotasint1[1,...]lowread-only
ssl.endpoint.identification.algorithmThe endpoint identification algorithm to validate server hostname using server certificate. stringhttpslowper-broker
ssl.secure.random.implementationThe SecureRandom PRNG implementation to use for SSL cryptography operations. stringnulllowper-broker
transaction.abort.timed.out.transaction.cleanup.interval.msThe interval at which to rollback transactions that have timed outint60000[1,...]lowread-only
transaction.remove.expired.transaction.cleanup.interval.msThe interval at which to remove transactions that have expired due to transactional.id.expiration.ms passingint3600000[1,...]lowread-only
zookeeper.sync.time.msHow far a ZK follower can be behind a ZK leaderint2000lowread-only
diff --git a/21/generated/producer_config.html b/21/generated/producer_config.html index e69de29..79108e2 100644 --- a/21/generated/producer_config.html +++ b/21/generated/producer_config.html @@ -0,0 +1,134 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + [...] + + + + + + + + + + + + + + + +< [...] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
key.serializerSerializer class for key that implements the org.apache.kafka.common.serialization.Serializer interface.classhigh
value.serializerSerializer class for value that implements the org.apache.kafka.common.serialization.Serializer interface.classhigh
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. This controls the durability of records that are sent. The following settings are allowed:
  • acks=0 If set to zero then the producer will not wait for any acknowledgment from the server at all. The record will be immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received th [...] +
bootstrap.serversA list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form host1:port1,host2:port2,.... Since these servers are just used for the initial connection to discover the full cluster me [...] +
buffer.memoryThe total bytes of memory the producer can use to buffer records waiting to be sent to the server. If records are sent faster than they can be delivered to the server the producer will block for max.block.ms after which it will throw an exception.

This setting should correspond roughly to the total memory the producer will use, but is not a hard bound since not all memory the producer uses is used for buffering. Some additional memory will be [...] +

compression.typeThe compression type for all data generated by the producer. The default is none (i.e. no compression). Valid values are none, gzip, snappy, lz4, or zstd. Compression is of full batches of data, so the efficacy of batching will also impact the compression ratio (more batching means better compression).stringnonehigh
retriesSetting a value greater than zero will cause the client to resend any record whose send fails with a potentially transient error. Note that this retry is no different than if the client resent the record upon receiving the error. Allowing retries without setting max.in.flight.requests.per.connection to 1 will potentially change the ordering of records because if two batches are sent to a single partition, and the first fails and is retried but the se [...] +
ssl.key.passwordThe password of the private key in the key store file. This is optional for client.passwordnullhigh
ssl.keystore.locationThe location of the key store file. This is optional for client and can be used for two-way authentication for client.stringnullhigh
ssl.keystore.passwordThe store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured. passwordnullhigh
ssl.truststore.locationThe location of the trust store file. stringnullhigh
ssl.truststore.passwordThe password for the trust store file. If a password is not set access to the truststore is still available, but integrity checking is disabled.passwordnullhigh
batch.sizeThe producer will attempt to batch records together into fewer requests whenever multiple records are being sent to the same partition. This helps performance on both the client and the server. This configuration controls the default batch size in bytes.

No attempt will be made to batch records larger than this size.

Requests sent to brokers will contain multiple batches, one for each partition with data available to be sent.

A small batch size will m [...] +

client.dns.lookup

Controls how the client uses DNS lookups.

If set to use_all_dns_ips then, when the lookup returns multiple IP addresses for a hostname, they will all be attempted to connect to before failing the connection. Applies to both bootstrap and advertised servers.

If the value is resolve_canonical_bootstrap_servers_only each entry will be resolved and expanded into a list of canonical names.

string
client.idAn id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.string""medium
connections.max.idle.msClose idle connections after the number of milliseconds specified by this config.long540000medium
delivery.timeout.msAn upper bound on the time to report success or failure after a call to send() returns. This limits the total time that a record will be delayed prior to sending, the time to await acknowledgement from the broker (if expected), and the time allowed for retriable send failures. The producer may report failure to send a record earlier than this config if either an unrecoverable error is encountered, the retries have been exhausted, or the r [...] +
linger.msThe producer groups together any records that arrive in between request transmissions into a single batched request. Normally this occurs only under load when records arrive faster than they can be sent out. However in some circumstances the client may want to reduce the number of requests even under moderate load. This setting accomplishes this by adding a small amount of artificial delay—that is, rather than immediately sending out a record the producer [...] +
max.block.msThe configuration controls how long KafkaProducer.send() and KafkaProducer.partitionsFor() will block.These methods can be blocked either because the buffer is full or metadata unavailable.Blocking in the user-supplied serializers or partitioner will not be counted against this timeout.long60000[0,...]medium
max.request.sizeThe maximum size of a request in bytes. This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests. This is also effectively a cap on the maximum record batch size. Note that the server has its own cap on record batch size which may be different from this.int1048576[0,...]medium
partitioner.classPartitioner class that implements the org.apache.kafka.clients.producer.Partitioner interface.classorg.apache.kafka.clients.producer.internals.DefaultPartitionermedium
receive.buffer.bytesThe size of the TCP receive buffer (SO_RCVBUF) to use when reading data. If the value is -1, the OS default will be used.int32768[-1,...]medium
request.timeout.msThe configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted. This should be larger than replica.lag.time.max.ms (a broker configuration) to reduce the possibility of message duplication due to unnecessary producer retries.int
sasl.client.callback.handler.classThe fully qualified name of a SASL client callback handler class that implements the AuthenticateCallbackHandler interface.classnullmedium
sasl.jaas.configJAAS login context parameters for SASL connections in the format used by JAAS configuration files. JAAS configuration file format is described here. The format for the value is: 'loginModuleClass controlFlag (optionName=optionValue)*;'. For brokers, the config must be prefixed with listener prefix and SASL mechanism name in lower- [...] +
sasl.kerberos.service.nameThe Kerberos principal name that Kafka runs as. This can be defined either in Kafka's JAAS config or in Kafka's config.stringnullmedium
sasl.login.callback.handler.classThe fully qualified name of a SASL login callback handler class that implements the AuthenticateCallbackHandler interface. For brokers, login callback handler config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.callback.handler.class=com.example.CustomScramLoginCallbackHandlerclassnullmedium [...] +
sasl.login.classThe fully qualified name of a class that implements the Login interface. For brokers, login config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.class=com.example.CustomScramLoginclassnullmedium
sasl.mechanismSASL mechanism used for client connections. This may be any mechanism for which a security provider is available. GSSAPI is the default mechanism.stringGSSAPImedium
security.protocolProtocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.stringPLAINTEXTmedium
send.buffer.bytesThe size of the TCP send buffer (SO_SNDBUF) to use when sending data. If the value is -1, the OS default will be used.int131072[-1,...]medium
ssl.enabled.protocolsThe list of protocols enabled for SSL connections.listTLSv1.2,TLSv1.1,TLSv1medium
ssl.keystore.typeThe file format of the key store file. This is optional for client.stringJKSmedium
ssl.protocolThe SSL protocol used to generate the SSLContext. Default setting is TLS, which is fine for most cases. Allowed values in recent JVMs are TLS, TLSv1.1 and TLSv1.2. SSL, SSLv2 and SSLv3 may be supported in older JVMs, but their usage is discouraged due to known security vulnerabilities.stringTLSmedium
ssl.providerThe name of the security provider used for SSL connections. Default value is the default security provider of the JVM.stringnullmedium
ssl.truststore.typeThe file format of the trust store file.stringJKSmedium
enable.idempotenceWhen set to 'true', the producer will ensure that exactly one copy of each message is written in the stream. If 'false', producer retries due to broker failures, etc., may write duplicates of the retried message in the stream. Note that enabling idempotence requires max.in.flight.requests.per.connection to be less than or equal to 5, retries to be greater than 0 and acks must be 'all'. If these values are not expl [...] +
interceptor.classesA list of classes to use as interceptors. Implementing the org.apache.kafka.clients.producer.ProducerInterceptor interface allows you to intercept (and possibly mutate) the records received by the producer before they are published to the Kafka cluster. By default, there are no interceptors.list""non-null stringlow
max.in.flight.requests.per.connectionThe maximum number of unacknowledged requests the client will send on a single connection before blocking. Note that if this setting is set to be greater than 1 and there are failed sends, there is a risk of message re-ordering due to retries (i.e., if retries are enabled).int5[1,...]low
metadata.max.age.msThe period of time in milliseconds after which we force a refresh of metadata even if we haven't seen any partition leadership changes to proactively discover any new brokers or partitions.long300000[0,...]low
metric.reportersA list of classes to use as metrics reporters. Implementing the org.apache.kafka.common.metrics.MetricsReporter interface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics.list""non-null stringlow
metrics.num.samplesThe number of samples maintained to compute metrics.int2[1,...]low
metrics.recording.levelThe highest recording level for metrics.stringINFO[INFO, DEBUG]low
metrics.sample.window.msThe window of time a metrics sample is computed over.long30000[0,...]low
reconnect.backoff.max.msThe maximum amount of time in milliseconds to wait when reconnecting to a broker that has repeatedly failed to connect. If provided, the backoff per host will increase exponentially for each consecutive connection failure, up to this maximum. After calculating the backoff increase, 20% random jitter is added to avoid connection storms.long1000[0,...]low
reconnect.backoff.msThe base amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop. This backoff applies to all connection attempts by the client to a broker.long50[0,...]low
retry.backoff.msThe amount of time to wait before attempting to retry a failed request to a given topic partition. This avoids repeatedly sending requests in a tight loop under some failure scenarios.long100[0,...]low
sasl.kerberos.kinit.cmdKerberos kinit command path.string/usr/bin/kinitlow
sasl.kerberos.min.time.before.reloginLogin thread sleep time between refresh attempts.long60000low
sasl.kerberos.ticket.renew.jitterPercentage of random jitter added to the renewal time.double0.05low
sasl.kerberos.ticket.renew.window.factorLogin thread will sleep until the specified window factor of time from last refresh to ticket's expiry has been reached, at which time it will try to renew the ticket.double0.8low
sasl.login.refresh.buffer.secondsThe amount of buffer time before credential expiration to maintain when refreshing a credential, in seconds. If a refresh would otherwise occur closer to expiration than the number of buffer seconds then the refresh will be moved up to maintain as much of the buffer time as possible. Legal values are between 0 and 3600 (1 hour); a default value of 300 (5 minutes) is used if no value is specified. This value and sasl.login.refresh.min.pe [...] +
sasl.login.refresh.min.period.secondsThe desired minimum time for the login refresh thread to wait before refreshing a credential, in seconds. Legal values are between 0 and 900 (15 minutes); a default value of 60 (1 minute) is used if no value is specified. This value and sasl.login.refresh.buffer.seconds are both ignored if their sum exceeds the remaining lifetime of a credential. Currently applies only to OAUTHBEARER.short60 +sasl.login.refresh.window.factorLogin refresh thread will sleep until the specified window factor relative to the credential's lifetime has been reached, at which time it will try to refresh the credential. Legal values are between 0.5 (50%) and 1.0 (100%) inclusive; a default value of 0.8 (80%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.8[0.5,...,1.0]low
sasl.login.refresh.window.jitterThe maximum amount of random jitter relative to the credential's lifetime that is added to the login refresh thread's sleep time. Legal values are between 0 and 0.25 (25%) inclusive; a default value of 0.05 (5%) is used if no value is specified. Currently applies only to OAUTHBEARER.double0.05[0.0,...,0.25]low
ssl.cipher.suitesA list of cipher suites. This is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. By default all the available cipher suites are supported.listnulllow
ssl.endpoint.identification.algorithmThe endpoint identification algorithm to validate server hostname using server certificate. stringhttpslow
ssl.keymanager.algorithmThe algorithm used by key manager factory for SSL connections. Default value is the key manager factory algorithm configured for the Java Virtual Machine.stringSunX509low
ssl.secure.random.implementationThe SecureRandom PRNG implementation to use for SSL cryptography operations. stringnulllow
ssl.trustmanager.algorithmThe algorithm used by trust manager factory for SSL connections. Default value is the trust manager factory algorithm configured for the Java Virtual Machine.stringPKIXlow
transaction.timeout.msThe maximum amount of time in ms that the transaction coordinator will wait for a transaction status update from the producer before proactively aborting the ongoing transaction.If this value is larger than the transaction.max.timeout.ms setting in the broker, the request will fail with a InvalidTransactionTimeout error.int60000low
transactional.idThe TransactionalId to use for transactional delivery. This enables reliability semantics which span multiple producer sessions since it allows the client to guarantee that transactions using the same TransactionalId have been completed prior to starting any new transactions. If no TransactionalId is provided, then the producer is limited to idempotent delivery. Note that enable.idempotence must be enabled if a TransactionalId is configured. [...] +
diff --git a/21/generated/producer_metrics.html b/21/generated/producer_metrics.html index e69de29..deb4078 100644 --- a/21/generated/producer_metrics.html +++ b/21/generated/producer_metrics.html @@ -0,0 +1,78 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
kafka.producer:type=producer-metrics,client-id="{client-id}"
Attribute nameDescription
batch-size-avgThe average number of bytes sent per partition per-request.
batch-size-maxThe max number of bytes sent per partition per-request.
batch-split-rateThe average number of batch splits per second
batch-split-totalThe total number of batch splits
compression-rate-avgThe average compression rate of record batches.
metadata-ageThe age in seconds of the current producer metadata being used.
produce-throttle-time-avgThe average time in ms a request was throttled by a broker
produce-throttle-time-maxThe maximum time in ms a request was throttled by a broker
record-error-rateThe average per-second number of record sends that resulted in errors
record-error-totalThe total number of record sends that resulted in errors
record-queue-time-avgThe average time in ms record batches spent in the send buffer.
record-queue-time-maxThe maximum time in ms record batches spent in the send buffer.
record-retry-rateThe average per-second number of retried record sends
record-retry-totalThe total number of retried record sends
record-send-rateThe average number of records sent per second.
record-send-totalThe total number of records sent.
record-size-avgThe average record size
record-size-maxThe maximum record size
records-per-request-avgThe average number of records per request.
request-latency-avgThe average request latency in ms
request-latency-maxThe maximum request latency in ms
requests-in-flightThe current number of in-flight requests awaiting a response.
kafka.producer:type=producer-topic-metrics,client-id="{client-id}",topic="{topic}"
Attribute nameDescription
byte-rateThe average number of bytes sent per second for a topic.
byte-totalThe total number of bytes sent for a topic.
compression-rateThe average compression rate of record batches for a topic.
record-error-rateThe average per-second number of record sends that resulted in errors for a topic
record-error-totalThe total number of record sends that resulted in errors for a topic
record-retry-rateThe average per-second number of retried record sends for a topic
record-retry-totalThe total number of retried record sends for a topic
record-send-rateThe average number of records sent per second for a topic.
record-send-totalThe total number of records sent for a topic.
diff --git a/21/generated/protocol_api_keys.html b/21/generated/protocol_api_keys.html index e69de29..923716a 100644 --- a/21/generated/protocol_api_keys.html +++ b/21/generated/protocol_api_keys.html @@ -0,0 +1,91 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameKey
Produce0
Fetch1
ListOffsets2
Metadata3
LeaderAndIsr4
StopReplica5
UpdateMetadata6
ControlledShutdown7
OffsetCommit8
OffsetFetch9
FindCoordinator10
JoinGroup11
Heartbeat12
LeaveGroup13
SyncGroup14
DescribeGroups15
ListGroups16
SaslHandshake17
ApiVersions18
CreateTopics19
DeleteTopics20
DeleteRecords21
InitProducerId22
OffsetForLeaderEpoch23
AddPartitionsToTxn24
AddOffsetsToTxn25
EndTxn26
WriteTxnMarkers27
TxnOffsetCommit28
DescribeAcls29
CreateAcls30
DeleteAcls31
DescribeConfigs32
AlterConfigs33
AlterReplicaLogDirs34
DescribeLogDirs35
SaslAuthenticate36
CreatePartitions37
CreateDelegationToken38
RenewDelegationToken39
ExpireDelegationToken40
DescribeDelegationToken41
DeleteGroups42
+ diff --git a/21/generated/protocol_errors.html b/21/generated/protocol_errors.html index e69de29..a85c14b 100644 --- a/21/generated/protocol_errors.html +++ b/21/generated/protocol_errors.html @@ -0,0 +1,86 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ErrorCodeRetriableDescription
UNKNOWN_SERVER_ERROR-1FalseThe server experienced an unexpected error when processing the request.
NONE0False
OFFSET_OUT_OF_RANGE1FalseThe requested offset is not within the range of offsets maintained by the server.
CORRUPT_MESSAGE2TrueThis message has failed its CRC checksum, exceeds the valid size, has a null key for a compacted topic, or is otherwise corrupt.
UNKNOWN_TOPIC_OR_PARTITION3TrueThis server does not host this topic-partition.
INVALID_FETCH_SIZE4FalseThe requested fetch size is invalid.
LEADER_NOT_AVAILABLE5TrueThere is no leader for this topic-partition as we are in the middle of a leadership election.
NOT_LEADER_FOR_PARTITION6TrueThis server is not the leader for that topic-partition.
REQUEST_TIMED_OUT7TrueThe request timed out.
BROKER_NOT_AVAILABLE8FalseThe broker is not available.
REPLICA_NOT_AVAILABLE9FalseThe replica is not available for the requested topic-partition.
MESSAGE_TOO_LARGE10FalseThe request included a message larger than the max message size the server will accept.
STALE_CONTROLLER_EPOCH11FalseThe controller moved to another broker.
OFFSET_METADATA_TOO_LARGE12FalseThe metadata field of the offset request was too large.
NETWORK_EXCEPTION13TrueThe server disconnected before a response was received.
COORDINATOR_LOAD_IN_PROGRESS14TrueThe coordinator is loading and hence can't process requests.
COORDINATOR_NOT_AVAILABLE15TrueThe coordinator is not available.
NOT_COORDINATOR16TrueThis is not the correct coordinator.
INVALID_TOPIC_EXCEPTION17FalseThe request attempted to perform an operation on an invalid topic.
RECORD_LIST_TOO_LARGE18FalseThe request included message batch larger than the configured segment size on the server.
NOT_ENOUGH_REPLICAS19TrueMessages are rejected since there are fewer in-sync replicas than required.
NOT_ENOUGH_REPLICAS_AFTER_APPEND20TrueMessages are written to the log, but to fewer in-sync replicas than required.
INVALID_REQUIRED_ACKS21FalseProduce request specified an invalid value for required acks.
ILLEGAL_GENERATION22FalseSpecified group generation id is not valid.
INCONSISTENT_GROUP_PROTOCOL23FalseThe group member's supported protocols are incompatible with those of existing members or first group member tried to join with empty protocol type or empty protocol list.
INVALID_GROUP_ID24FalseThe configured groupId is invalid.
UNKNOWN_MEMBER_ID25FalseThe coordinator is not aware of this member.
INVALID_SESSION_TIMEOUT26FalseThe session timeout is not within the range allowed by the broker (as configured by group.min.session.timeout.ms and group.max.session.timeout.ms).
REBALANCE_IN_PROGRESS27FalseThe group is rebalancing, so a rejoin is needed.
INVALID_COMMIT_OFFSET_SIZE28FalseThe committing offset data size is not valid.
TOPIC_AUTHORIZATION_FAILED29FalseNot authorized to access topics: [Topic authorization failed.]
GROUP_AUTHORIZATION_FAILED30FalseNot authorized to access group: Group authorization failed.
CLUSTER_AUTHORIZATION_FAILED31FalseCluster authorization failed.
INVALID_TIMESTAMP32FalseThe timestamp of the message is out of acceptable range.
UNSUPPORTED_SASL_MECHANISM33FalseThe broker does not support the requested SASL mechanism.
ILLEGAL_SASL_STATE34FalseRequest is not valid given the current SASL state.
UNSUPPORTED_VERSION35FalseThe version of API is not supported.
TOPIC_ALREADY_EXISTS36FalseTopic with this name already exists.
INVALID_PARTITIONS37FalseNumber of partitions is below 1.
INVALID_REPLICATION_FACTOR38FalseReplication factor is below 1 or larger than the number of available brokers.
INVALID_REPLICA_ASSIGNMENT39FalseReplica assignment is invalid.
INVALID_CONFIG40FalseConfiguration is invalid.
NOT_CONTROLLER41TrueThis is not the correct controller for this cluster.
INVALID_REQUEST42FalseThis most likely occurs because of a request being malformed by the client library or the message was sent to an incompatible broker. See the broker logs for more details.
UNSUPPORTED_FOR_MESSAGE_FORMAT43FalseThe message format version on the broker does not support the request.
POLICY_VIOLATION44FalseRequest parameters do not satisfy the configured policy.
OUT_OF_ORDER_SEQUENCE_NUMBER45FalseThe broker received an out of order sequence number.
DUPLICATE_SEQUENCE_NUMBER46FalseThe broker received a duplicate sequence number.
INVALID_PRODUCER_EPOCH47FalseProducer attempted an operation with an old epoch. Either there is a newer producer with the same transactionalId, or the producer's transaction has been expired by the broker.
INVALID_TXN_STATE48FalseThe producer attempted a transactional operation in an invalid state.
INVALID_PRODUCER_ID_MAPPING49FalseThe producer attempted to use a producer id which is not currently assigned to its transactional id.
INVALID_TRANSACTION_TIMEOUT50FalseThe transaction timeout is larger than the maximum value allowed by the broker (as configured by transaction.max.timeout.ms).
CONCURRENT_TRANSACTIONS51FalseThe producer attempted to update a transaction while another concurrent operation on the same transaction was ongoing.
TRANSACTION_COORDINATOR_FENCED52FalseIndicates that the transaction coordinator sending a WriteTxnMarker is no longer the current coordinator for a given producer.
TRANSACTIONAL_ID_AUTHORIZATION_FAILED53FalseTransactional Id authorization failed.
SECURITY_DISABLED54FalseSecurity features are disabled.
OPERATION_NOT_ATTEMPTED55FalseThe broker did not attempt to execute this operation. This may happen for batched RPCs where some operations in the batch failed, causing the broker to respond without trying the rest.
KAFKA_STORAGE_ERROR56TrueDisk error when trying to access log file on the disk.
LOG_DIR_NOT_FOUND57FalseThe user-specified log directory is not found in the broker config.
SASL_AUTHENTICATION_FAILED58FalseSASL Authentication failed.
UNKNOWN_PRODUCER_ID59FalseThis exception is raised by the broker if it could not locate the producer metadata associated with the producerId in question. This could happen if, for instance, the producer's records were deleted because their retention time had elapsed. Once the last records of the producerId are removed, the producer's metadata is removed from the broker, and future appends by the producer will return this exception.
REASSIGNMENT_IN_PROGRESS60FalseA partition reassignment is in progress.
DELEGATION_TOKEN_AUTH_DISABLED61FalseDelegation Token feature is not enabled.
DELEGATION_TOKEN_NOT_FOUND62FalseDelegation Token is not found on server.
DELEGATION_TOKEN_OWNER_MISMATCH63FalseSpecified Principal is not valid Owner/Renewer.
DELEGATION_TOKEN_REQUEST_NOT_ALLOWED64FalseDelegation Token requests are not allowed on PLAINTEXT/1-way SSL channels and on delegation token authenticated channels.
DELEGATION_TOKEN_AUTHORIZATION_FAILED65FalseDelegation Token authorization failed.
DELEGATION_TOKEN_EXPIRED66FalseDelegation Token is expired.
INVALID_PRINCIPAL_TYPE67FalseSupplied principalType is not supported.
NON_EMPTY_GROUP68FalseThe group is not empty.
GROUP_ID_NOT_FOUND69FalseThe group id does not exist.
FETCH_SESSION_ID_NOT_FOUND70TrueThe fetch session ID was not found.
INVALID_FETCH_SESSION_EPOCH71TrueThe fetch session epoch is invalid.
LISTENER_NOT_FOUND72TrueThere is no listener on the leader broker that matches the listener on which metadata request was processed.
TOPIC_DELETION_DISABLED73FalseTopic deletion is disabled.
FENCED_LEADER_EPOCH74TrueThe leader epoch in the request is older than the epoch on the broker
UNKNOWN_LEADER_EPOCH75TrueThe leader epoch in the request is newer than the epoch on the broker
UNSUPPORTED_COMPRESSION_TYPE76FalseThe requesting client does not support the compression type of given partition.
+ diff --git a/21/generated/protocol_messages.html b/21/generated/protocol_messages.html index e69de29..17a16d0 100644 --- a/21/generated/protocol_messages.html +++ b/21/generated/protocol_messages.html @@ -0,0 +1,7281 @@ +
Headers:
+
Request Header => api_key api_version correlation_id client_id 
+  api_key => INT16
+  api_version => INT16
+  correlation_id => INT32
+  client_id => NULLABLE_STRING
+
+ + + + + + + + + + + +
FieldDescription
api_keyThe id of the request type.
api_versionThe version of the API.
correlation_idA user-supplied integer value that will be passed back with the response
client_idA user specified identifier for the client making the request.
+
Response Header => correlation_id 
+  correlation_id => INT32
+
+ + + + + +
FieldDescription
correlation_idThe user-supplied value passed in with the request
+
Produce API (Key: 0):
+ +Requests:
+

Produce Request (Version: 0) => acks timeout [topic_data] 
+  acks => INT16
+  timeout => INT32
+  topic_data => topic [data] 
+    topic => STRING
+    data => partition record_set 
+      partition => INT32
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + +
FieldDescription
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.
timeoutThe time to await a response in ms.
topic_datanull
topicName of topic
datanull
partitionTopic partition id
record_setnull
+

+

Produce Request (Version: 1) => acks timeout [topic_data] 
+  acks => INT16
+  timeout => INT32
+  topic_data => topic [data] 
+    topic => STRING
+    data => partition record_set 
+      partition => INT32
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + +
FieldDescription
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.
timeoutThe time to await a response in ms.
topic_datanull
topicName of topic
datanull
partitionTopic partition id
record_setnull
+

+

Produce Request (Version: 2) => acks timeout [topic_data] 
+  acks => INT16
+  timeout => INT32
+  topic_data => topic [data] 
+    topic => STRING
+    data => partition record_set 
+      partition => INT32
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + +
FieldDescription
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.
timeoutThe time to await a response in ms.
topic_datanull
topicName of topic
datanull
partitionTopic partition id
record_setnull
+

+

Produce Request (Version: 3) => transactional_id acks timeout [topic_data] 
+  transactional_id => NULLABLE_STRING
+  acks => INT16
+  timeout => INT32
+  topic_data => topic [data] 
+    topic => STRING
+    data => partition record_set 
+      partition => INT32
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id or null if the producer is not transactional
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.
timeoutThe time to await a response in ms.
topic_datanull
topicName of topic
datanull
partitionTopic partition id
record_setnull
+

+

Produce Request (Version: 4) => transactional_id acks timeout [topic_data] 
+  transactional_id => NULLABLE_STRING
+  acks => INT16
+  timeout => INT32
+  topic_data => topic [data] 
+    topic => STRING
+    data => partition record_set 
+      partition => INT32
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id or null if the producer is not transactional
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.
timeoutThe time to await a response in ms.
topic_datanull
topicName of topic
datanull
partitionTopic partition id
record_setnull
+

+

Produce Request (Version: 5) => transactional_id acks timeout [topic_data] 
+  transactional_id => NULLABLE_STRING
+  acks => INT16
+  timeout => INT32
+  topic_data => topic [data] 
+    topic => STRING
+    data => partition record_set 
+      partition => INT32
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id or null if the producer is not transactional
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.
timeoutThe time to await a response in ms.
topic_datanull
topicName of topic
datanull
partitionTopic partition id
record_setnull
+

+

Produce Request (Version: 6) => transactional_id acks timeout [topic_data] 
+  transactional_id => NULLABLE_STRING
+  acks => INT16
+  timeout => INT32
+  topic_data => topic [data] 
+    topic => STRING
+    data => partition record_set 
+      partition => INT32
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id or null if the producer is not transactional
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.
timeoutThe time to await a response in ms.
topic_datanull
topicName of topic
datanull
partitionTopic partition id
record_setnull
+

+

Produce Request (Version: 7) => transactional_id acks timeout [topic_data] 
+  transactional_id => NULLABLE_STRING
+  acks => INT16
+  timeout => INT32
+  topic_data => topic [data] 
+    topic => STRING
+    data => partition record_set 
+      partition => INT32
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id or null if the producer is not transactional
acksThe number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.
timeoutThe time to await a response in ms.
topic_datanull
topicName of topic
datanull
partitionTopic partition id
record_setnull
+

+Responses:
+

Produce Response (Version: 0) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code base_offset 
+      partition => INT32
+      error_code => INT16
+      base_offset => INT64
+
+ + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partitionTopic partition id
error_codeResponse error code
base_offsetnull
+

+

Produce Response (Version: 1) => [responses] throttle_time_ms 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code base_offset 
+      partition => INT32
+      error_code => INT16
+      base_offset => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partitionTopic partition id
error_codeResponse error code
base_offsetnull
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

Produce Response (Version: 2) => [responses] throttle_time_ms 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code base_offset log_append_time 
+      partition => INT32
+      error_code => INT16
+      base_offset => INT64
+      log_append_time => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partitionTopic partition id
error_codeResponse error code
base_offsetnull
log_append_timeThe timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended.
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

Produce Response (Version: 3) => [responses] throttle_time_ms 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code base_offset log_append_time 
+      partition => INT32
+      error_code => INT16
+      base_offset => INT64
+      log_append_time => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partitionTopic partition id
error_codeResponse error code
base_offsetnull
log_append_timeThe timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended.
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

Produce Response (Version: 4) => [responses] throttle_time_ms 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code base_offset log_append_time 
+      partition => INT32
+      error_code => INT16
+      base_offset => INT64
+      log_append_time => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partitionTopic partition id
error_codeResponse error code
base_offsetnull
log_append_timeThe timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended.
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

Produce Response (Version: 5) => [responses] throttle_time_ms 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code base_offset log_append_time log_start_offset 
+      partition => INT32
+      error_code => INT16
+      base_offset => INT64
+      log_append_time => INT64
+      log_start_offset => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partitionTopic partition id
error_codeResponse error code
base_offsetnull
log_append_timeThe timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended.
log_start_offsetThe start offset of the log at the time this produce response was created
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

Produce Response (Version: 6) => [responses] throttle_time_ms 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code base_offset log_append_time log_start_offset 
+      partition => INT32
+      error_code => INT16
+      base_offset => INT64
+      log_append_time => INT64
+      log_start_offset => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partitionTopic partition id
error_codeResponse error code
base_offsetnull
log_append_timeThe timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended.
log_start_offsetThe start offset of the log at the time this produce response was created
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

Produce Response (Version: 7) => [responses] throttle_time_ms 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code base_offset log_append_time log_start_offset 
+      partition => INT32
+      error_code => INT16
+      base_offset => INT64
+      log_append_time => INT64
+      log_start_offset => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partitionTopic partition id
error_codeResponse error code
base_offsetnull
log_append_timeThe timestamp returned by broker after appending the messages. If CreateTime is used for the topic, the timestamp will be -1. If LogAppendTime is used for the topic, the timestamp will be the broker local time when the messages are appended.
log_start_offsetThe start offset of the log at the time this produce response was created
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+
Fetch API (Key: 1):
+ +Requests:
+

Fetch Request (Version: 0) => replica_id max_wait_time min_bytes [topics] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      partition_max_bytes => INT32
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
partition_max_bytesMaximum bytes to fetch.
+

+

Fetch Request (Version: 1) => replica_id max_wait_time min_bytes [topics] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      partition_max_bytes => INT32
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
partition_max_bytesMaximum bytes to fetch.
+

+

Fetch Request (Version: 2) => replica_id max_wait_time min_bytes [topics] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      partition_max_bytes => INT32
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
partition_max_bytesMaximum bytes to fetch.
+

+

Fetch Request (Version: 3) => replica_id max_wait_time min_bytes max_bytes [topics] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  max_bytes => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      partition_max_bytes => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
max_bytesMaximum bytes to accumulate in the response. Note that this is not an absolute maximum, if the first message in the first non-empty partition of the fetch is larger than this value, the message will still be returned to ensure that progress can be made.
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
partition_max_bytesMaximum bytes to fetch.
+

+

Fetch Request (Version: 4) => replica_id max_wait_time min_bytes max_bytes isolation_level [topics] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  max_bytes => INT32
+  isolation_level => INT8
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      partition_max_bytes => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
max_bytesMaximum bytes to accumulate in the response. Note that this is not an absolute maximum, if the first message in the first non-empty partition of the fetch is larger than this value, the message will still be returned to ensure that progress can be made.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
partition_max_bytesMaximum bytes to fetch.
+

+

Fetch Request (Version: 5) => replica_id max_wait_time min_bytes max_bytes isolation_level [topics] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  max_bytes => INT32
+  isolation_level => INT8
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset log_start_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      log_start_offset => INT64
+      partition_max_bytes => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
max_bytesMaximum bytes to accumulate in the response. Note that this is not an absolute maximum, if the first message in the first non-empty partition of the fetch is larger than this value, the message will still be returned to ensure that progress can be made.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
log_start_offsetEarliest available offset of the follower replica. The field is only used when request is sent by follower.
partition_max_bytesMaximum bytes to fetch.
+

+

Fetch Request (Version: 6) => replica_id max_wait_time min_bytes max_bytes isolation_level [topics] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  max_bytes => INT32
+  isolation_level => INT8
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset log_start_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      log_start_offset => INT64
+      partition_max_bytes => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
max_bytesMaximum bytes to accumulate in the response. Note that this is not an absolute maximum, if the first message in the first non-empty partition of the fetch is larger than this value, the message will still be returned to ensure that progress can be made.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
log_start_offsetEarliest available offset of the follower replica. The field is only used when request is sent by follower.
partition_max_bytesMaximum bytes to fetch.
+

+

Fetch Request (Version: 7) => replica_id max_wait_time min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  max_bytes => INT32
+  isolation_level => INT8
+  session_id => INT32
+  session_epoch => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset log_start_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      log_start_offset => INT64
+      partition_max_bytes => INT32
+  forgotten_topics_data => topic [partitions] 
+    topic => STRING
+    partitions => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
max_bytesMaximum bytes to accumulate in the response. Note that this is not an absolute maximum, if the first message in the first non-empty partition of the fetch is larger than this value, the message will still be returned to ensure that progress can be made.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
session_idThe fetch session ID
session_epochThe fetch session epoch
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
log_start_offsetEarliest available offset of the follower replica. The field is only used when request is sent by follower.
partition_max_bytesMaximum bytes to fetch.
forgotten_topics_dataTopics to remove from the fetch session.
topicName of topic
partitionsPartitions to remove from the fetch session.
+

+

Fetch Request (Version: 8) => replica_id max_wait_time min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  max_bytes => INT32
+  isolation_level => INT8
+  session_id => INT32
+  session_epoch => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition fetch_offset log_start_offset partition_max_bytes 
+      partition => INT32
+      fetch_offset => INT64
+      log_start_offset => INT64
+      partition_max_bytes => INT32
+  forgotten_topics_data => topic [partitions] 
+    topic => STRING
+    partitions => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
max_bytesMaximum bytes to accumulate in the response. Note that this is not an absolute maximum, if the first message in the first non-empty partition of the fetch is larger than this value, the message will still be returned to ensure that progress can be made.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
session_idThe fetch session ID
session_epochThe fetch session epoch
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
fetch_offsetMessage offset.
log_start_offsetEarliest available offset of the follower replica. The field is only used when request is sent by follower.
partition_max_bytesMaximum bytes to fetch.
forgotten_topics_dataTopics to remove from the fetch session.
topicName of topic
partitionsPartitions to remove from the fetch session.
+

+

Fetch Request (Version: 9) => replica_id max_wait_time min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  max_bytes => INT32
+  isolation_level => INT8
+  session_id => INT32
+  session_epoch => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition current_leader_epoch fetch_offset log_start_offset partition_max_bytes 
+      partition => INT32
+      current_leader_epoch => INT32
+      fetch_offset => INT64
+      log_start_offset => INT64
+      partition_max_bytes => INT32
+  forgotten_topics_data => topic [partitions] 
+    topic => STRING
+    partitions => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
max_bytesMaximum bytes to accumulate in the response. Note that this is not an absolute maximum, if the first message in the first non-empty partition of the fetch is larger than this value, the message will still be returned to ensure that progress can be made.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
session_idThe fetch session ID
session_epochThe fetch session epoch
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
current_leader_epochThe current leader epoch, if provided, is used to fence consumers/replicas with old metadata. If the epoch provided by the client is larger than the current epoch known to the broker, then the UNKNOWN_LEADER_EPOCH error code will be returned. If the provided epoch is smaller, then the FENCED_LEADER_EPOCH error code will be returned.
fetch_offsetMessage offset.
log_start_offsetEarliest available offset of the follower replica. The field is only used when request is sent by follower.
partition_max_bytesMaximum bytes to fetch.
forgotten_topics_dataTopics to remove from the fetch session.
topicName of topic
partitionsPartitions to remove from the fetch session.
+

+

Fetch Request (Version: 10) => replica_id max_wait_time min_bytes max_bytes isolation_level session_id session_epoch [topics] [forgotten_topics_data] 
+  replica_id => INT32
+  max_wait_time => INT32
+  min_bytes => INT32
+  max_bytes => INT32
+  isolation_level => INT8
+  session_id => INT32
+  session_epoch => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition current_leader_epoch fetch_offset log_start_offset partition_max_bytes 
+      partition => INT32
+      current_leader_epoch => INT32
+      fetch_offset => INT64
+      log_start_offset => INT64
+      partition_max_bytes => INT32
+  forgotten_topics_data => topic [partitions] 
+    topic => STRING
+    partitions => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
max_wait_timeMaximum time in ms to wait for the response.
min_bytesMinimum bytes to accumulate in the response.
max_bytesMaximum bytes to accumulate in the response. Note that this is not an absolute maximum, if the first message in the first non-empty partition of the fetch is larger than this value, the message will still be returned to ensure that progress can be made.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
session_idThe fetch session ID
session_epochThe fetch session epoch
topicsTopics to fetch in the order provided.
topicName of topic
partitionsPartitions to fetch.
partitionTopic partition id
current_leader_epochThe current leader epoch, if provided, is used to fence consumers/replicas with old metadata. If the epoch provided by the client is larger than the current epoch known to the broker, then the UNKNOWN_LEADER_EPOCH error code will be returned. If the provided epoch is smaller, then the FENCED_LEADER_EPOCH error code will be returned.
fetch_offsetMessage offset.
log_start_offsetEarliest available offset of the follower replica. The field is only used when request is sent by follower.
partition_max_bytesMaximum bytes to fetch.
forgotten_topics_dataTopics to remove from the fetch session.
topicName of topic
partitionsPartitions to remove from the fetch session.
+

+Responses:
+

Fetch Response (Version: 0) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
record_setnull
+

+

Fetch Response (Version: 1) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
record_setnull
+

+

Fetch Response (Version: 2) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
record_setnull
+

+

Fetch Response (Version: 3) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
record_setnull
+

+

Fetch Response (Version: 4) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark last_stable_offset [aborted_transactions] 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+        last_stable_offset => INT64
+        aborted_transactions => producer_id first_offset 
+          producer_id => INT64
+          first_offset => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
last_stable_offsetThe last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED)
aborted_transactionsnull
producer_idThe producer id associated with the aborted transactions
first_offsetThe first offset in the aborted transaction
record_setnull
+

+

Fetch Response (Version: 5) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+        last_stable_offset => INT64
+        log_start_offset => INT64
+        aborted_transactions => producer_id first_offset 
+          producer_id => INT64
+          first_offset => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
last_stable_offsetThe last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED)
log_start_offsetEarliest available offset.
aborted_transactionsnull
producer_idThe producer id associated with the aborted transactions
first_offsetThe first offset in the aborted transaction
record_setnull
+

+

Fetch Response (Version: 6) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+        last_stable_offset => INT64
+        log_start_offset => INT64
+        aborted_transactions => producer_id first_offset 
+          producer_id => INT64
+          first_offset => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
last_stable_offsetThe last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED)
log_start_offsetEarliest available offset.
aborted_transactionsnull
producer_idThe producer id associated with the aborted transactions
first_offsetThe first offset in the aborted transaction
record_setnull
+

+

Fetch Response (Version: 7) => throttle_time_ms error_code session_id [responses] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  session_id => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+        last_stable_offset => INT64
+        log_start_offset => INT64
+        aborted_transactions => producer_id first_offset 
+          producer_id => INT64
+          first_offset => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
session_idThe fetch session ID
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
last_stable_offsetThe last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED)
log_start_offsetEarliest available offset.
aborted_transactionsnull
producer_idThe producer id associated with the aborted transactions
first_offsetThe first offset in the aborted transaction
record_setnull
+

+

Fetch Response (Version: 8) => throttle_time_ms error_code session_id [responses] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  session_id => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+        last_stable_offset => INT64
+        log_start_offset => INT64
+        aborted_transactions => producer_id first_offset 
+          producer_id => INT64
+          first_offset => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
session_idThe fetch session ID
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
last_stable_offsetThe last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED)
log_start_offsetEarliest available offset.
aborted_transactionsnull
producer_idThe producer id associated with the aborted transactions
first_offsetThe first offset in the aborted transaction
record_setnull
+

+

Fetch Response (Version: 9) => throttle_time_ms error_code session_id [responses] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  session_id => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+        last_stable_offset => INT64
+        log_start_offset => INT64
+        aborted_transactions => producer_id first_offset 
+          producer_id => INT64
+          first_offset => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
session_idThe fetch session ID
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
last_stable_offsetThe last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED)
log_start_offsetEarliest available offset.
aborted_transactionsnull
producer_idThe producer id associated with the aborted transactions
first_offsetThe first offset in the aborted transaction
record_setnull
+

+

Fetch Response (Version: 10) => throttle_time_ms error_code session_id [responses] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  session_id => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition_header record_set 
+      partition_header => partition error_code high_watermark last_stable_offset log_start_offset [aborted_transactions] 
+        partition => INT32
+        error_code => INT16
+        high_watermark => INT64
+        last_stable_offset => INT64
+        log_start_offset => INT64
+        aborted_transactions => producer_id first_offset 
+          producer_id => INT64
+          first_offset => INT64
+      record_set => RECORDS
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
session_idThe fetch session ID
responsesnull
topicName of topic
partition_responsesnull
partition_headernull
partitionTopic partition id
error_codeResponse error code
high_watermarkLast committed offset.
last_stable_offsetThe last stable offset (or LSO) of the partition. This is the last offset such that the state of all transactional records prior to this offset have been decided (ABORTED or COMMITTED)
log_start_offsetEarliest available offset.
aborted_transactionsnull
producer_idThe producer id associated with the aborted transactions
first_offsetThe first offset in the aborted transaction
record_setnull
+

+
ListOffsets API (Key: 2):
+ +Requests:
+

ListOffsets Request (Version: 0) => replica_id [topics] 
+  replica_id => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition timestamp max_num_offsets 
+      partition => INT32
+      timestamp => INT64
+      max_num_offsets => INT32
+
+ + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
topicsTopics to list offsets.
topicName of topic
partitionsPartitions to list offsets.
partitionTopic partition id
timestampThe target timestamp for the partition.
max_num_offsetsMaximum offsets to return.
+

+

ListOffsets Request (Version: 1) => replica_id [topics] 
+  replica_id => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition timestamp 
+      partition => INT32
+      timestamp => INT64
+
+ + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
topicsTopics to list offsets.
topicName of topic
partitionsPartitions to list offsets.
partitionTopic partition id
timestampThe target timestamp for the partition.
+

+

ListOffsets Request (Version: 2) => replica_id isolation_level [topics] 
+  replica_id => INT32
+  isolation_level => INT8
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition timestamp 
+      partition => INT32
+      timestamp => INT64
+
+ + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
topicsTopics to list offsets.
topicName of topic
partitionsPartitions to list offsets.
partitionTopic partition id
timestampThe target timestamp for the partition.
+

+

ListOffsets Request (Version: 3) => replica_id isolation_level [topics] 
+  replica_id => INT32
+  isolation_level => INT8
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition timestamp 
+      partition => INT32
+      timestamp => INT64
+
+ + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
topicsTopics to list offsets.
topicName of topic
partitionsPartitions to list offsets.
partitionTopic partition id
timestampThe target timestamp for the partition.
+

+

ListOffsets Request (Version: 4) => replica_id isolation_level [topics] 
+  replica_id => INT32
+  isolation_level => INT8
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition current_leader_epoch timestamp 
+      partition => INT32
+      current_leader_epoch => INT32
+      timestamp => INT64
+
+ + + + + + + + + + + + + + + + + +
FieldDescription
replica_idBroker id of the follower. For normal consumers, use -1.
isolation_levelThis setting controls the visibility of transactional records. Using READ_UNCOMMITTED (isolation_level = 0) makes all records visible. With READ_COMMITTED (isolation_level = 1), non-transactional and COMMITTED transactional records are visible. To be more concrete, READ_COMMITTED returns all data from offsets smaller than the current LSO (last stable offset), and enables the inclusion of the list of aborted transactions in the result, which allows consumers to [...] +
topicsTopics to list offsets.
topicName of topic
partitionsPartitions to list offsets.
partitionTopic partition id
current_leader_epochThe current leader epoch, if provided, is used to fence consumers/replicas with old metadata. If the epoch provided by the client is larger than the current epoch known to the broker, then the UNKNOWN_LEADER_EPOCH error code will be returned. If the provided epoch is smaller, then the FENCED_LEADER_EPOCH error code will be returned.
timestampThe target timestamp for the partition.
+

+Responses:
+

ListOffsets Response (Version: 0) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code [offsets'] 
+      partition => INT32
+      error_code => INT16
+      offsets' => INT64
+
+ + + + + + + + + + + + + + +
FieldDescription
responsesThe listed offsets by topic
topicName of topic
partition_responsesThe listed offsets by partition
partitionTopic partition id
error_codeResponse error code
offsets'A list of offsets.
+

+

ListOffsets Response (Version: 1) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code timestamp offset 
+      partition => INT32
+      error_code => INT16
+      timestamp => INT64
+      offset => INT64
+
+ + + + + + + + + + + + + + + + +
FieldDescription
responsesThe listed offsets by topic
topicName of topic
partition_responsesThe listed offsets by partition
partitionTopic partition id
error_codeResponse error code
timestampThe timestamp associated with the returned offset
offsetThe offset found
+

+

ListOffsets Response (Version: 2) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code timestamp offset 
+      partition => INT32
+      error_code => INT16
+      timestamp => INT64
+      offset => INT64
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesThe listed offsets by topic
topicName of topic
partition_responsesThe listed offsets by partition
partitionTopic partition id
error_codeResponse error code
timestampThe timestamp associated with the returned offset
offsetThe offset found
+

+

ListOffsets Response (Version: 3) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code timestamp offset 
+      partition => INT32
+      error_code => INT16
+      timestamp => INT64
+      offset => INT64
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesThe listed offsets by topic
topicName of topic
partition_responsesThe listed offsets by partition
partitionTopic partition id
error_codeResponse error code
timestampThe timestamp associated with the returned offset
offsetThe offset found
+

+

ListOffsets Response (Version: 4) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code timestamp offset leader_epoch 
+      partition => INT32
+      error_code => INT16
+      timestamp => INT64
+      offset => INT64
+      leader_epoch => INT32
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesThe listed offsets by topic
topicName of topic
partition_responsesThe listed offsets by partition
partitionTopic partition id
error_codeResponse error code
timestampThe timestamp associated with the returned offset
offsetThe offset found
leader_epochThe leader epoch
+

+
Metadata API (Key: 3):
+ +Requests:
+

Metadata Request (Version: 0) => [topics] 
+  topics => STRING
+
+ + + + +
FieldDescription
topicsAn array of topics to fetch metadata for. If no topics are specified fetch metadata for all topics.
+

+

Metadata Request (Version: 1) => [topics] 
+  topics => STRING
+
+ + + + +
FieldDescription
topicsAn array of topics to fetch metadata for. If the topics array is null fetch metadata for all topics.
+

+

Metadata Request (Version: 2) => [topics] 
+  topics => STRING
+
+ + + + +
FieldDescription
topicsAn array of topics to fetch metadata for. If the topics array is null fetch metadata for all topics.
+

+

Metadata Request (Version: 3) => [topics] 
+  topics => STRING
+
+ + + + +
FieldDescription
topicsAn array of topics to fetch metadata for. If the topics array is null fetch metadata for all topics.
+

+

Metadata Request (Version: 4) => [topics] allow_auto_topic_creation 
+  topics => STRING
+  allow_auto_topic_creation => BOOLEAN
+
+ + + + + + +
FieldDescription
topicsAn array of topics to fetch metadata for. If the topics array is null fetch metadata for all topics.
allow_auto_topic_creationIf this and the broker config auto.create.topics.enable are true, topics that don't exist will be created by the broker. Otherwise, no topics will be created by the broker.
+

+

Metadata Request (Version: 5) => [topics] allow_auto_topic_creation 
+  topics => STRING
+  allow_auto_topic_creation => BOOLEAN
+
+ + + + + + +
FieldDescription
topicsAn array of topics to fetch metadata for. If the topics array is null fetch metadata for all topics.
allow_auto_topic_creationIf this and the broker config auto.create.topics.enable are true, topics that don't exist will be created by the broker. Otherwise, no topics will be created by the broker.
+

+

Metadata Request (Version: 6) => [topics] allow_auto_topic_creation 
+  topics => STRING
+  allow_auto_topic_creation => BOOLEAN
+
+ + + + + + +
FieldDescription
topicsAn array of topics to fetch metadata for. If the topics array is null fetch metadata for all topics.
allow_auto_topic_creationIf this and the broker config auto.create.topics.enable are true, topics that don't exist will be created by the broker. Otherwise, no topics will be created by the broker.
+

+

Metadata Request (Version: 7) => [topics] allow_auto_topic_creation 
+  topics => STRING
+  allow_auto_topic_creation => BOOLEAN
+
+ + + + + + +
FieldDescription
topicsAn array of topics to fetch metadata for. If the topics array is null fetch metadata for all topics.
allow_auto_topic_creationIf this and the broker config auto.create.topics.enable are true, topics that don't exist will be created by the broker. Otherwise, no topics will be created by the broker.
+

+Responses:
+

Metadata Response (Version: 0) => [brokers] [topic_metadata] 
+  brokers => node_id host port 
+    node_id => INT32
+    host => STRING
+    port => INT32
+  topic_metadata => error_code topic [partition_metadata] 
+    error_code => INT16
+    topic => STRING
+    partition_metadata => error_code partition leader [replicas] [isr] 
+      error_code => INT16
+      partition => INT32
+      leader => INT32
+      replicas => INT32
+      isr => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
brokersHost and port information for all brokers.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
topic_metadataMetadata for requested topics
error_codeResponse error code
topicName of topic
partition_metadataMetadata for each partition of the topic.
error_codeResponse error code
partitionTopic partition id
leaderThe id of the broker acting as leader for this partition.
replicasThe set of all nodes that host this partition.
isrThe set of nodes that are in sync with the leader for this partition.
+

+

Metadata Response (Version: 1) => [brokers] controller_id [topic_metadata] 
+  brokers => node_id host port rack 
+    node_id => INT32
+    host => STRING
+    port => INT32
+    rack => NULLABLE_STRING
+  controller_id => INT32
+  topic_metadata => error_code topic is_internal [partition_metadata] 
+    error_code => INT16
+    topic => STRING
+    is_internal => BOOLEAN
+    partition_metadata => error_code partition leader [replicas] [isr] 
+      error_code => INT16
+      partition => INT32
+      leader => INT32
+      replicas => INT32
+      isr => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
brokersHost and port information for all brokers.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
rackThe rack of the broker.
controller_idThe broker id of the controller broker.
topic_metadataMetadata for requested topics
error_codeResponse error code
topicName of topic
is_internalIndicates if the topic is considered a Kafka internal topic
partition_metadataMetadata for each partition of the topic.
error_codeResponse error code
partitionTopic partition id
leaderThe id of the broker acting as leader for this partition.
replicasThe set of all nodes that host this partition.
isrThe set of nodes that are in sync with the leader for this partition.
+

+

Metadata Response (Version: 2) => [brokers] cluster_id controller_id [topic_metadata] 
+  brokers => node_id host port rack 
+    node_id => INT32
+    host => STRING
+    port => INT32
+    rack => NULLABLE_STRING
+  cluster_id => NULLABLE_STRING
+  controller_id => INT32
+  topic_metadata => error_code topic is_internal [partition_metadata] 
+    error_code => INT16
+    topic => STRING
+    is_internal => BOOLEAN
+    partition_metadata => error_code partition leader [replicas] [isr] 
+      error_code => INT16
+      partition => INT32
+      leader => INT32
+      replicas => INT32
+      isr => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
brokersHost and port information for all brokers.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
rackThe rack of the broker.
cluster_idThe cluster id that this broker belongs to.
controller_idThe broker id of the controller broker.
topic_metadataMetadata for requested topics
error_codeResponse error code
topicName of topic
is_internalIndicates if the topic is considered a Kafka internal topic
partition_metadataMetadata for each partition of the topic.
error_codeResponse error code
partitionTopic partition id
leaderThe id of the broker acting as leader for this partition.
replicasThe set of all nodes that host this partition.
isrThe set of nodes that are in sync with the leader for this partition.
+

+

Metadata Response (Version: 3) => throttle_time_ms [brokers] cluster_id controller_id [topic_metadata] 
+  throttle_time_ms => INT32
+  brokers => node_id host port rack 
+    node_id => INT32
+    host => STRING
+    port => INT32
+    rack => NULLABLE_STRING
+  cluster_id => NULLABLE_STRING
+  controller_id => INT32
+  topic_metadata => error_code topic is_internal [partition_metadata] 
+    error_code => INT16
+    topic => STRING
+    is_internal => BOOLEAN
+    partition_metadata => error_code partition leader [replicas] [isr] 
+      error_code => INT16
+      partition => INT32
+      leader => INT32
+      replicas => INT32
+      isr => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
brokersHost and port information for all brokers.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
rackThe rack of the broker.
cluster_idThe cluster id that this broker belongs to.
controller_idThe broker id of the controller broker.
topic_metadataMetadata for requested topics
error_codeResponse error code
topicName of topic
is_internalIndicates if the topic is considered a Kafka internal topic
partition_metadataMetadata for each partition of the topic.
error_codeResponse error code
partitionTopic partition id
leaderThe id of the broker acting as leader for this partition.
replicasThe set of all nodes that host this partition.
isrThe set of nodes that are in sync with the leader for this partition.
+

+

Metadata Response (Version: 4) => throttle_time_ms [brokers] cluster_id controller_id [topic_metadata] 
+  throttle_time_ms => INT32
+  brokers => node_id host port rack 
+    node_id => INT32
+    host => STRING
+    port => INT32
+    rack => NULLABLE_STRING
+  cluster_id => NULLABLE_STRING
+  controller_id => INT32
+  topic_metadata => error_code topic is_internal [partition_metadata] 
+    error_code => INT16
+    topic => STRING
+    is_internal => BOOLEAN
+    partition_metadata => error_code partition leader [replicas] [isr] 
+      error_code => INT16
+      partition => INT32
+      leader => INT32
+      replicas => INT32
+      isr => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
brokersHost and port information for all brokers.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
rackThe rack of the broker.
cluster_idThe cluster id that this broker belongs to.
controller_idThe broker id of the controller broker.
topic_metadataMetadata for requested topics
error_codeResponse error code
topicName of topic
is_internalIndicates if the topic is considered a Kafka internal topic
partition_metadataMetadata for each partition of the topic.
error_codeResponse error code
partitionTopic partition id
leaderThe id of the broker acting as leader for this partition.
replicasThe set of all nodes that host this partition.
isrThe set of nodes that are in sync with the leader for this partition.
+

+

Metadata Response (Version: 5) => throttle_time_ms [brokers] cluster_id controller_id [topic_metadata] 
+  throttle_time_ms => INT32
+  brokers => node_id host port rack 
+    node_id => INT32
+    host => STRING
+    port => INT32
+    rack => NULLABLE_STRING
+  cluster_id => NULLABLE_STRING
+  controller_id => INT32
+  topic_metadata => error_code topic is_internal [partition_metadata] 
+    error_code => INT16
+    topic => STRING
+    is_internal => BOOLEAN
+    partition_metadata => error_code partition leader [replicas] [isr] [offline_replicas] 
+      error_code => INT16
+      partition => INT32
+      leader => INT32
+      replicas => INT32
+      isr => INT32
+      offline_replicas => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
brokersHost and port information for all brokers.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
rackThe rack of the broker.
cluster_idThe cluster id that this broker belongs to.
controller_idThe broker id of the controller broker.
topic_metadataMetadata for requested topics
error_codeResponse error code
topicName of topic
is_internalIndicates if the topic is considered a Kafka internal topic
partition_metadataMetadata for each partition of the topic.
error_codeResponse error code
partitionTopic partition id
leaderThe id of the broker acting as leader for this partition.
replicasThe set of all nodes that host this partition.
isrThe set of nodes that are in sync with the leader for this partition.
offline_replicasThe set of offline replicas of this partition.
+

+

Metadata Response (Version: 6) => throttle_time_ms [brokers] cluster_id controller_id [topic_metadata] 
+  throttle_time_ms => INT32
+  brokers => node_id host port rack 
+    node_id => INT32
+    host => STRING
+    port => INT32
+    rack => NULLABLE_STRING
+  cluster_id => NULLABLE_STRING
+  controller_id => INT32
+  topic_metadata => error_code topic is_internal [partition_metadata] 
+    error_code => INT16
+    topic => STRING
+    is_internal => BOOLEAN
+    partition_metadata => error_code partition leader [replicas] [isr] [offline_replicas] 
+      error_code => INT16
+      partition => INT32
+      leader => INT32
+      replicas => INT32
+      isr => INT32
+      offline_replicas => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
brokersHost and port information for all brokers.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
rackThe rack of the broker.
cluster_idThe cluster id that this broker belongs to.
controller_idThe broker id of the controller broker.
topic_metadataMetadata for requested topics
error_codeResponse error code
topicName of topic
is_internalIndicates if the topic is considered a Kafka internal topic
partition_metadataMetadata for each partition of the topic.
error_codeResponse error code
partitionTopic partition id
leaderThe id of the broker acting as leader for this partition.
replicasThe set of all nodes that host this partition.
isrThe set of nodes that are in sync with the leader for this partition.
offline_replicasThe set of offline replicas of this partition.
+

+

Metadata Response (Version: 7) => throttle_time_ms [brokers] cluster_id controller_id [topic_metadata] 
+  throttle_time_ms => INT32
+  brokers => node_id host port rack 
+    node_id => INT32
+    host => STRING
+    port => INT32
+    rack => NULLABLE_STRING
+  cluster_id => NULLABLE_STRING
+  controller_id => INT32
+  topic_metadata => error_code topic is_internal [partition_metadata] 
+    error_code => INT16
+    topic => STRING
+    is_internal => BOOLEAN
+    partition_metadata => error_code partition leader leader_epoch [replicas] [isr] [offline_replicas] 
+      error_code => INT16
+      partition => INT32
+      leader => INT32
+      leader_epoch => INT32
+      replicas => INT32
+      isr => INT32
+      offline_replicas => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
brokersHost and port information for all brokers.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
rackThe rack of the broker.
cluster_idThe cluster id that this broker belongs to.
controller_idThe broker id of the controller broker.
topic_metadataMetadata for requested topics
error_codeResponse error code
topicName of topic
is_internalIndicates if the topic is considered a Kafka internal topic
partition_metadataMetadata for each partition of the topic.
error_codeResponse error code
partitionTopic partition id
leaderThe id of the broker acting as leader for this partition.
leader_epochThe leader epoch
replicasThe set of all nodes that host this partition.
isrThe set of nodes that are in sync with the leader for this partition.
offline_replicasThe set of offline replicas of this partition.
+

+
LeaderAndIsr API (Key: 4):
+ +Requests:
+

LeaderAndIsr Request (Version: 0) => controller_id controller_epoch [partition_states] [live_leaders] 
+  controller_id => INT32
+  controller_epoch => INT32
+  partition_states => topic partition controller_epoch leader leader_epoch [isr] zk_version [replicas] 
+    topic => STRING
+    partition => INT32
+    controller_epoch => INT32
+    leader => INT32
+    leader_epoch => INT32
+    isr => INT32
+    zk_version => INT32
+    replicas => INT32
+  live_leaders => id host port 
+    id => INT32
+    host => STRING
+    port => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
controller_idThe controller id.
controller_epochThe controller epoch.
partition_statesnull
topicName of topic
partitionTopic partition id
controller_epochThe controller epoch.
leaderThe broker id for the leader.
leader_epochThe leader epoch.
isrThe in sync replica ids.
zk_versionThe ZK version.
replicasThe replica ids.
live_leadersnull
idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
+

+

LeaderAndIsr Request (Version: 1) => controller_id controller_epoch [partition_states] [live_leaders] 
+  controller_id => INT32
+  controller_epoch => INT32
+  partition_states => topic partition controller_epoch leader leader_epoch [isr] zk_version [replicas] is_new 
+    topic => STRING
+    partition => INT32
+    controller_epoch => INT32
+    leader => INT32
+    leader_epoch => INT32
+    isr => INT32
+    zk_version => INT32
+    replicas => INT32
+    is_new => BOOLEAN
+  live_leaders => id host port 
+    id => INT32
+    host => STRING
+    port => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
controller_idThe controller id.
controller_epochThe controller epoch.
partition_statesnull
topicName of topic
partitionTopic partition id
controller_epochThe controller epoch.
leaderThe broker id for the leader.
leader_epochThe leader epoch.
isrThe in sync replica ids.
zk_versionThe ZK version.
replicasThe replica ids.
is_newWhether the replica should have existed on the broker or not
live_leadersnull
idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
+

+Responses:
+

LeaderAndIsr Response (Version: 0) => error_code [partitions] 
+  error_code => INT16
+  partitions => topic partition error_code 
+    topic => STRING
+    partition => INT32
+    error_code => INT16
+
+ + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
partitionsnull
topicName of topic
partitionTopic partition id
error_codeResponse error code
+

+

LeaderAndIsr Response (Version: 1) => error_code [partitions] 
+  error_code => INT16
+  partitions => topic partition error_code 
+    topic => STRING
+    partition => INT32
+    error_code => INT16
+
+ + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
partitionsnull
topicName of topic
partitionTopic partition id
error_codeResponse error code
+

+
StopReplica API (Key: 5):
+ +Requests:
+

StopReplica Request (Version: 0) => controller_id controller_epoch delete_partitions [partitions] 
+  controller_id => INT32
+  controller_epoch => INT32
+  delete_partitions => BOOLEAN
+  partitions => topic partition 
+    topic => STRING
+    partition => INT32
+
+ + + + + + + + + + + + + + +
FieldDescription
controller_idThe controller id.
controller_epochThe controller epoch.
delete_partitionsBoolean which indicates if replica's partitions must be deleted.
partitionsnull
topicName of topic
partitionTopic partition id
+

+Responses:
+

StopReplica Response (Version: 0) => error_code [partitions] 
+  error_code => INT16
+  partitions => topic partition error_code 
+    topic => STRING
+    partition => INT32
+    error_code => INT16
+
+ + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
partitionsnull
topicName of topic
partitionTopic partition id
error_codeResponse error code
+

+
UpdateMetadata API (Key: 6):
+ +Requests:
+

UpdateMetadata Request (Version: 0) => controller_id controller_epoch [partition_states] [live_brokers] 
+  controller_id => INT32
+  controller_epoch => INT32
+  partition_states => topic partition controller_epoch leader leader_epoch [isr] zk_version [replicas] 
+    topic => STRING
+    partition => INT32
+    controller_epoch => INT32
+    leader => INT32
+    leader_epoch => INT32
+    isr => INT32
+    zk_version => INT32
+    replicas => INT32
+  live_brokers => id host port 
+    id => INT32
+    host => STRING
+    port => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
controller_idThe controller id.
controller_epochThe controller epoch.
partition_statesnull
topicName of topic
partitionTopic partition id
controller_epochThe controller epoch.
leaderThe broker id for the leader.
leader_epochThe leader epoch.
isrThe in sync replica ids.
zk_versionThe ZK version.
replicasThe replica ids.
live_brokersnull
idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
+

+

UpdateMetadata Request (Version: 1) => controller_id controller_epoch [partition_states] [live_brokers] 
+  controller_id => INT32
+  controller_epoch => INT32
+  partition_states => topic partition controller_epoch leader leader_epoch [isr] zk_version [replicas] 
+    topic => STRING
+    partition => INT32
+    controller_epoch => INT32
+    leader => INT32
+    leader_epoch => INT32
+    isr => INT32
+    zk_version => INT32
+    replicas => INT32
+  live_brokers => id [end_points] 
+    id => INT32
+    end_points => port host security_protocol_type 
+      port => INT32
+      host => STRING
+      security_protocol_type => INT16
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
controller_idThe controller id.
controller_epochThe controller epoch.
partition_statesnull
topicName of topic
partitionTopic partition id
controller_epochThe controller epoch.
leaderThe broker id for the leader.
leader_epochThe leader epoch.
isrThe in sync replica ids.
zk_versionThe ZK version.
replicasThe replica ids.
live_brokersnull
idThe broker id.
end_pointsnull
portThe port on which the broker accepts requests.
hostThe hostname of the broker.
security_protocol_typeThe security protocol type.
+

+

UpdateMetadata Request (Version: 2) => controller_id controller_epoch [partition_states] [live_brokers] 
+  controller_id => INT32
+  controller_epoch => INT32
+  partition_states => topic partition controller_epoch leader leader_epoch [isr] zk_version [replicas] 
+    topic => STRING
+    partition => INT32
+    controller_epoch => INT32
+    leader => INT32
+    leader_epoch => INT32
+    isr => INT32
+    zk_version => INT32
+    replicas => INT32
+  live_brokers => id [end_points] rack 
+    id => INT32
+    end_points => port host security_protocol_type 
+      port => INT32
+      host => STRING
+      security_protocol_type => INT16
+    rack => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
controller_idThe controller id.
controller_epochThe controller epoch.
partition_statesnull
topicName of topic
partitionTopic partition id
controller_epochThe controller epoch.
leaderThe broker id for the leader.
leader_epochThe leader epoch.
isrThe in sync replica ids.
zk_versionThe ZK version.
replicasThe replica ids.
live_brokersnull
idThe broker id.
end_pointsnull
portThe port on which the broker accepts requests.
hostThe hostname of the broker.
security_protocol_typeThe security protocol type.
rackThe rack
+

+

UpdateMetadata Request (Version: 3) => controller_id controller_epoch [partition_states] [live_brokers] 
+  controller_id => INT32
+  controller_epoch => INT32
+  partition_states => topic partition controller_epoch leader leader_epoch [isr] zk_version [replicas] 
+    topic => STRING
+    partition => INT32
+    controller_epoch => INT32
+    leader => INT32
+    leader_epoch => INT32
+    isr => INT32
+    zk_version => INT32
+    replicas => INT32
+  live_brokers => id [end_points] rack 
+    id => INT32
+    end_points => port host listener_name security_protocol_type 
+      port => INT32
+      host => STRING
+      listener_name => STRING
+      security_protocol_type => INT16
+    rack => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
controller_idThe controller id.
controller_epochThe controller epoch.
partition_statesnull
topicName of topic
partitionTopic partition id
controller_epochThe controller epoch.
leaderThe broker id for the leader.
leader_epochThe leader epoch.
isrThe in sync replica ids.
zk_versionThe ZK version.
replicasThe replica ids.
live_brokersnull
idThe broker id.
end_pointsnull
portThe port on which the broker accepts requests.
hostThe hostname of the broker.
listener_nameThe listener name.
security_protocol_typeThe security protocol type.
rackThe rack
+

+

UpdateMetadata Request (Version: 4) => controller_id controller_epoch [partition_states] [live_brokers] 
+  controller_id => INT32
+  controller_epoch => INT32
+  partition_states => topic partition controller_epoch leader leader_epoch [isr] zk_version [replicas] [offline_replicas] 
+    topic => STRING
+    partition => INT32
+    controller_epoch => INT32
+    leader => INT32
+    leader_epoch => INT32
+    isr => INT32
+    zk_version => INT32
+    replicas => INT32
+    offline_replicas => INT32
+  live_brokers => id [end_points] rack 
+    id => INT32
+    end_points => port host listener_name security_protocol_type 
+      port => INT32
+      host => STRING
+      listener_name => STRING
+      security_protocol_type => INT16
+    rack => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
controller_idThe controller id.
controller_epochThe controller epoch.
partition_statesnull
topicName of topic
partitionTopic partition id
controller_epochThe controller epoch.
leaderThe broker id for the leader.
leader_epochThe leader epoch.
isrThe in sync replica ids.
zk_versionThe ZK version.
replicasThe replica ids.
offline_replicasThe offline replica ids
live_brokersnull
idThe broker id.
end_pointsnull
portThe port on which the broker accepts requests.
hostThe hostname of the broker.
listener_nameThe listener name.
security_protocol_typeThe security protocol type.
rackThe rack
+

+Responses:
+

UpdateMetadata Response (Version: 0) => error_code 
+  error_code => INT16
+
+ + + + +
FieldDescription
error_codeResponse error code
+

+

UpdateMetadata Response (Version: 1) => error_code 
+  error_code => INT16
+
+ + + + +
FieldDescription
error_codeResponse error code
+

+

UpdateMetadata Response (Version: 2) => error_code 
+  error_code => INT16
+
+ + + + +
FieldDescription
error_codeResponse error code
+

+

UpdateMetadata Response (Version: 3) => error_code 
+  error_code => INT16
+
+ + + + +
FieldDescription
error_codeResponse error code
+

+

UpdateMetadata Response (Version: 4) => error_code 
+  error_code => INT16
+
+ + + + +
FieldDescription
error_codeResponse error code
+

+
ControlledShutdown API (Key: 7):
+ +Requests:
+

ControlledShutdown Request (Version: 0) => broker_id 
+  broker_id => INT32
+
+ + + + +
FieldDescription
broker_idThe id of the broker for which controlled shutdown has been requested.
+

+

ControlledShutdown Request (Version: 1) => broker_id 
+  broker_id => INT32
+
+ + + + +
FieldDescription
broker_idThe id of the broker for which controlled shutdown has been requested.
+

+Responses:
+

ControlledShutdown Response (Version: 0) => error_code [partitions_remaining] 
+  error_code => INT16
+  partitions_remaining => topic partition 
+    topic => STRING
+    partition => INT32
+
+ + + + + + + + + + +
FieldDescription
error_codeResponse error code
partitions_remainingThe partitions that the broker still leads.
topicName of topic
partitionTopic partition id
+

+

ControlledShutdown Response (Version: 1) => error_code [partitions_remaining] 
+  error_code => INT16
+  partitions_remaining => topic partition 
+    topic => STRING
+    partition => INT32
+
+ + + + + + + + + + +
FieldDescription
error_codeResponse error code
partitions_remainingThe partitions that the broker still leads.
topicName of topic
partitionTopic partition id
+

+
OffsetCommit API (Key: 8):
+ +Requests:
+

OffsetCommit Request (Version: 0) => group_id [topics] 
+  group_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset metadata 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
+

+

OffsetCommit Request (Version: 1) => group_id generation_id member_id [topics] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset timestamp metadata 
+      partition => INT32
+      offset => INT64
+      timestamp => INT64
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
timestampTimestamp of the commit
metadataAny associated metadata the client wants to keep.
+

+

OffsetCommit Request (Version: 2) => group_id generation_id member_id retention_time [topics] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  retention_time => INT64
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset metadata 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
retention_timeTime period in ms to retain the offset.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
+

+

OffsetCommit Request (Version: 3) => group_id generation_id member_id retention_time [topics] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  retention_time => INT64
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset metadata 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
retention_timeTime period in ms to retain the offset.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
+

+

OffsetCommit Request (Version: 4) => group_id generation_id member_id retention_time [topics] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  retention_time => INT64
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset metadata 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
retention_timeTime period in ms to retain the offset.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
+

+

OffsetCommit Request (Version: 5) => group_id generation_id member_id [topics] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset metadata 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
+

+

OffsetCommit Request (Version: 6) => group_id generation_id member_id [topics] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset leader_epoch metadata 
+      partition => INT32
+      offset => INT64
+      leader_epoch => INT32
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
leader_epochThe leader epoch, if provided is derived from the last consumed record. This is used by the consumer to check for log truncation and to ensure partition metadata is up to date following a group rebalance.
metadataAny associated metadata the client wants to keep.
+

+Responses:
+

OffsetCommit Response (Version: 0) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + +
FieldDescription
responsesResponses by topic for committed partitions
topicName of topic
partition_responsesResponses for committed partitions
partitionTopic partition id
error_codeResponse error code
+

+

OffsetCommit Response (Version: 1) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + +
FieldDescription
responsesResponses by topic for committed partitions
topicName of topic
partition_responsesResponses for committed partitions
partitionTopic partition id
error_codeResponse error code
+

+

OffsetCommit Response (Version: 2) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + +
FieldDescription
responsesResponses by topic for committed partitions
topicName of topic
partition_responsesResponses for committed partitions
partitionTopic partition id
error_codeResponse error code
+

+

OffsetCommit Response (Version: 3) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesResponses by topic for committed partitions
topicName of topic
partition_responsesResponses for committed partitions
partitionTopic partition id
error_codeResponse error code
+

+

OffsetCommit Response (Version: 4) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesResponses by topic for committed partitions
topicName of topic
partition_responsesResponses for committed partitions
partitionTopic partition id
error_codeResponse error code
+

+

OffsetCommit Response (Version: 5) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesResponses by topic for committed partitions
topicName of topic
partition_responsesResponses for committed partitions
partitionTopic partition id
error_codeResponse error code
+

+

OffsetCommit Response (Version: 6) => throttle_time_ms [responses] 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesResponses by topic for committed partitions
topicName of topic
partition_responsesResponses for committed partitions
partitionTopic partition id
error_codeResponse error code
+

+
OffsetFetch API (Key: 9):
+ +Requests:
+

OffsetFetch Request (Version: 0) => group_id [topics] 
+  group_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition 
+      partition => INT32
+
+ + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
topicsTopics to fetch offsets.
topicName of topic
partitionsPartitions to fetch offsets.
partitionTopic partition id
+

+

OffsetFetch Request (Version: 1) => group_id [topics] 
+  group_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition 
+      partition => INT32
+
+ + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
topicsTopics to fetch offsets.
topicName of topic
partitionsPartitions to fetch offsets.
partitionTopic partition id
+

+

OffsetFetch Request (Version: 2) => group_id [topics] 
+  group_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition 
+      partition => INT32
+
+ + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
topicsTopics to fetch offsets. If the topic array is null fetch offsets for all topics.
topicName of topic
partitionsPartitions to fetch offsets.
partitionTopic partition id
+

+

OffsetFetch Request (Version: 3) => group_id [topics] 
+  group_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition 
+      partition => INT32
+
+ + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
topicsTopics to fetch offsets. If the topic array is null fetch offsets for all topics.
topicName of topic
partitionsPartitions to fetch offsets.
partitionTopic partition id
+

+

OffsetFetch Request (Version: 4) => group_id [topics] 
+  group_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition 
+      partition => INT32
+
+ + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
topicsTopics to fetch offsets. If the topic array is null fetch offsets for all topics.
topicName of topic
partitionsPartitions to fetch offsets.
partitionTopic partition id
+

+

OffsetFetch Request (Version: 5) => group_id [topics] 
+  group_id => STRING
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition 
+      partition => INT32
+
+ + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
topicsTopics to fetch offsets. If the topic array is null fetch offsets for all topics.
topicName of topic
partitionsPartitions to fetch offsets.
partitionTopic partition id
+

+Responses:
+

OffsetFetch Response (Version: 0) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition offset metadata error_code 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+      error_code => INT16
+
+ + + + + + + + + + + + + + + + +
FieldDescription
responsesResponses by topic for fetched offsets
topicName of topic
partition_responsesResponses by partition for fetched offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
error_codeResponse error code
+

+

OffsetFetch Response (Version: 1) => [responses] 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition offset metadata error_code 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+      error_code => INT16
+
+ + + + + + + + + + + + + + + + +
FieldDescription
responsesResponses by topic for fetched offsets
topicName of topic
partition_responsesResponses by partition for fetched offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
error_codeResponse error code
+

+

OffsetFetch Response (Version: 2) => [responses] error_code 
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition offset metadata error_code 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+      error_code => INT16
+  error_code => INT16
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
responsesResponses by topic for fetched offsets
topicName of topic
partition_responsesResponses by partition for fetched offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
error_codeResponse error code
error_codeResponse error code
+

+

OffsetFetch Response (Version: 3) => throttle_time_ms [responses] error_code 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition offset metadata error_code 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+      error_code => INT16
+  error_code => INT16
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesResponses by topic for fetched offsets
topicName of topic
partition_responsesResponses by partition for fetched offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
error_codeResponse error code
error_codeResponse error code
+

+

OffsetFetch Response (Version: 4) => throttle_time_ms [responses] error_code 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition offset metadata error_code 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+      error_code => INT16
+  error_code => INT16
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesResponses by topic for fetched offsets
topicName of topic
partition_responsesResponses by partition for fetched offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
error_codeResponse error code
error_codeResponse error code
+

+

OffsetFetch Response (Version: 5) => throttle_time_ms [responses] error_code 
+  throttle_time_ms => INT32
+  responses => topic [partition_responses] 
+    topic => STRING
+    partition_responses => partition offset leader_epoch metadata error_code 
+      partition => INT32
+      offset => INT64
+      leader_epoch => INT32
+      metadata => NULLABLE_STRING
+      error_code => INT16
+  error_code => INT16
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
responsesResponses by topic for fetched offsets
topicName of topic
partition_responsesResponses by partition for fetched offsets
partitionTopic partition id
offsetMessage offset to be committed
leader_epochThe leader epoch, if provided is derived from the last consumed record. This is used by the consumer to check for log truncation and to ensure partition metadata is up to date following a group rebalance.
metadataAny associated metadata the client wants to keep.
error_codeResponse error code
error_codeResponse error code
+

+
FindCoordinator API (Key: 10):
+ +Requests:
+

FindCoordinator Request (Version: 0) => group_id 
+  group_id => STRING
+
+ + + + +
FieldDescription
group_idThe unique group identifier
+

+

FindCoordinator Request (Version: 1) => coordinator_key coordinator_type 
+  coordinator_key => STRING
+  coordinator_type => INT8
+
+ + + + + + +
FieldDescription
coordinator_keyId to use for finding the coordinator (for groups, this is the groupId, for transactional producers, this is the transactional id)
coordinator_typeThe type of coordinator to find (0 = group, 1 = transaction)
+

+

FindCoordinator Request (Version: 2) => coordinator_key coordinator_type 
+  coordinator_key => STRING
+  coordinator_type => INT8
+
+ + + + + + +
FieldDescription
coordinator_keyId to use for finding the coordinator (for groups, this is the groupId, for transactional producers, this is the transactional id)
coordinator_typeThe type of coordinator to find (0 = group, 1 = transaction)
+

+Responses:
+

FindCoordinator Response (Version: 0) => error_code coordinator 
+  error_code => INT16
+  coordinator => node_id host port 
+    node_id => INT32
+    host => STRING
+    port => INT32
+
+ + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
coordinatorHost and port information for the coordinator for a consumer group.
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
+

+

FindCoordinator Response (Version: 1) => throttle_time_ms error_code error_message coordinator 
+  throttle_time_ms => INT32
+  error_code => INT16
+  error_message => NULLABLE_STRING
+  coordinator => node_id host port 
+    node_id => INT32
+    host => STRING
+    port => INT32
+
+ + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
error_messageResponse error message
coordinatorHost and port information for the coordinator
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
+

+

FindCoordinator Response (Version: 2) => throttle_time_ms error_code error_message coordinator 
+  throttle_time_ms => INT32
+  error_code => INT16
+  error_message => NULLABLE_STRING
+  coordinator => node_id host port 
+    node_id => INT32
+    host => STRING
+    port => INT32
+
+ + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
error_messageResponse error message
coordinatorHost and port information for the coordinator
node_idThe broker id.
hostThe hostname of the broker.
portThe port on which the broker accepts requests.
+

+
JoinGroup API (Key: 11):
+ +Requests:
+

JoinGroup Request (Version: 0) => group_id session_timeout member_id protocol_type [group_protocols] 
+  group_id => STRING
+  session_timeout => INT32
+  member_id => STRING
+  protocol_type => STRING
+  group_protocols => protocol_name protocol_metadata 
+    protocol_name => STRING
+    protocol_metadata => BYTES
+
+ + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
session_timeoutThe coordinator considers the consumer dead if it receives no heartbeat after this timeout in ms.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
protocol_typeUnique name for class of protocols implemented by group
group_protocolsList of protocols that the member supports
protocol_namenull
protocol_metadatanull
+

+

JoinGroup Request (Version: 1) => group_id session_timeout rebalance_timeout member_id protocol_type [group_protocols] 
+  group_id => STRING
+  session_timeout => INT32
+  rebalance_timeout => INT32
+  member_id => STRING
+  protocol_type => STRING
+  group_protocols => protocol_name protocol_metadata 
+    protocol_name => STRING
+    protocol_metadata => BYTES
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
session_timeoutThe coordinator considers the consumer dead if it receives no heartbeat after this timeout in ms.
rebalance_timeoutThe maximum time that the coordinator will wait for each member to rejoin when rebalancing the group
member_idThe member id assigned by the group coordinator or null if joining for the first time.
protocol_typeUnique name for class of protocols implemented by group
group_protocolsList of protocols that the member supports
protocol_namenull
protocol_metadatanull
+

+

JoinGroup Request (Version: 2) => group_id session_timeout rebalance_timeout member_id protocol_type [group_protocols] 
+  group_id => STRING
+  session_timeout => INT32
+  rebalance_timeout => INT32
+  member_id => STRING
+  protocol_type => STRING
+  group_protocols => protocol_name protocol_metadata 
+    protocol_name => STRING
+    protocol_metadata => BYTES
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
session_timeoutThe coordinator considers the consumer dead if it receives no heartbeat after this timeout in ms.
rebalance_timeoutThe maximum time that the coordinator will wait for each member to rejoin when rebalancing the group
member_idThe member id assigned by the group coordinator or null if joining for the first time.
protocol_typeUnique name for class of protocols implemented by group
group_protocolsList of protocols that the member supports
protocol_namenull
protocol_metadatanull
+

+

JoinGroup Request (Version: 3) => group_id session_timeout rebalance_timeout member_id protocol_type [group_protocols] 
+  group_id => STRING
+  session_timeout => INT32
+  rebalance_timeout => INT32
+  member_id => STRING
+  protocol_type => STRING
+  group_protocols => protocol_name protocol_metadata 
+    protocol_name => STRING
+    protocol_metadata => BYTES
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
session_timeoutThe coordinator considers the consumer dead if it receives no heartbeat after this timeout in ms.
rebalance_timeoutThe maximum time that the coordinator will wait for each member to rejoin when rebalancing the group
member_idThe member id assigned by the group coordinator or null if joining for the first time.
protocol_typeUnique name for class of protocols implemented by group
group_protocolsList of protocols that the member supports
protocol_namenull
protocol_metadatanull
+

+Responses:
+

JoinGroup Response (Version: 0) => error_code generation_id group_protocol leader_id member_id [members] 
+  error_code => INT16
+  generation_id => INT32
+  group_protocol => STRING
+  leader_id => STRING
+  member_id => STRING
+  members => member_id member_metadata 
+    member_id => STRING
+    member_metadata => BYTES
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
generation_idThe generation of the group.
group_protocolThe group protocol selected by the coordinator
leader_idThe leader of the group
member_idThe member id assigned by the group coordinator or null if joining for the first time.
membersnull
member_idThe member id assigned by the group coordinator or null if joining for the first time.
member_metadatanull
+

+

JoinGroup Response (Version: 1) => error_code generation_id group_protocol leader_id member_id [members] 
+  error_code => INT16
+  generation_id => INT32
+  group_protocol => STRING
+  leader_id => STRING
+  member_id => STRING
+  members => member_id member_metadata 
+    member_id => STRING
+    member_metadata => BYTES
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
generation_idThe generation of the group.
group_protocolThe group protocol selected by the coordinator
leader_idThe leader of the group
member_idThe member id assigned by the group coordinator or null if joining for the first time.
membersnull
member_idThe member id assigned by the group coordinator or null if joining for the first time.
member_metadatanull
+

+

JoinGroup Response (Version: 2) => throttle_time_ms error_code generation_id group_protocol leader_id member_id [members] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  generation_id => INT32
+  group_protocol => STRING
+  leader_id => STRING
+  member_id => STRING
+  members => member_id member_metadata 
+    member_id => STRING
+    member_metadata => BYTES
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
generation_idThe generation of the group.
group_protocolThe group protocol selected by the coordinator
leader_idThe leader of the group
member_idThe member id assigned by the group coordinator or null if joining for the first time.
membersnull
member_idThe member id assigned by the group coordinator or null if joining for the first time.
member_metadatanull
+

+

JoinGroup Response (Version: 3) => throttle_time_ms error_code generation_id group_protocol leader_id member_id [members] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  generation_id => INT32
+  group_protocol => STRING
+  leader_id => STRING
+  member_id => STRING
+  members => member_id member_metadata 
+    member_id => STRING
+    member_metadata => BYTES
+
+ + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
generation_idThe generation of the group.
group_protocolThe group protocol selected by the coordinator
leader_idThe leader of the group
member_idThe member id assigned by the group coordinator or null if joining for the first time.
membersnull
member_idThe member id assigned by the group coordinator or null if joining for the first time.
member_metadatanull
+

+
Heartbeat API (Key: 12):
+ +Requests:
+

Heartbeat Request (Version: 0) => group_id generation_id member_id 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+
+ + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
+

+

Heartbeat Request (Version: 1) => group_id generation_id member_id 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+
+ + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
+

+

Heartbeat Request (Version: 2) => group_id generation_id member_id 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+
+ + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
+

+Responses:
+

Heartbeat Response (Version: 0) => error_code 
+  error_code => INT16
+
+ + + + +
FieldDescription
error_codeResponse error code
+

+

Heartbeat Response (Version: 1) => throttle_time_ms error_code 
+  throttle_time_ms => INT32
+  error_code => INT16
+
+ + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
+

+

Heartbeat Response (Version: 2) => throttle_time_ms error_code 
+  throttle_time_ms => INT32
+  error_code => INT16
+
+ + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
+

+
LeaveGroup API (Key: 13):
+ +Requests:
+

LeaveGroup Request (Version: 0) => group_id member_id 
+  group_id => STRING
+  member_id => STRING
+
+ + + + + + +
FieldDescription
group_idThe unique group identifier
member_idThe member id assigned by the group coordinator or null if joining for the first time.
+

+

LeaveGroup Request (Version: 1) => group_id member_id 
+  group_id => STRING
+  member_id => STRING
+
+ + + + + + +
FieldDescription
group_idThe unique group identifier
member_idThe member id assigned by the group coordinator or null if joining for the first time.
+

+

LeaveGroup Request (Version: 2) => group_id member_id 
+  group_id => STRING
+  member_id => STRING
+
+ + + + + + +
FieldDescription
group_idThe unique group identifier
member_idThe member id assigned by the group coordinator or null if joining for the first time.
+

+Responses:
+

LeaveGroup Response (Version: 0) => error_code 
+  error_code => INT16
+
+ + + + +
FieldDescription
error_codeResponse error code
+

+

LeaveGroup Response (Version: 1) => throttle_time_ms error_code 
+  throttle_time_ms => INT32
+  error_code => INT16
+
+ + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
+

+

LeaveGroup Response (Version: 2) => throttle_time_ms error_code 
+  throttle_time_ms => INT32
+  error_code => INT16
+
+ + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
+

+
SyncGroup API (Key: 14):
+ +Requests:
+

SyncGroup Request (Version: 0) => group_id generation_id member_id [group_assignment] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  group_assignment => member_id member_assignment 
+    member_id => STRING
+    member_assignment => BYTES
+
+ + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
group_assignmentnull
member_idThe member id assigned by the group coordinator or null if joining for the first time.
member_assignmentnull
+

+

SyncGroup Request (Version: 1) => group_id generation_id member_id [group_assignment] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  group_assignment => member_id member_assignment 
+    member_id => STRING
+    member_assignment => BYTES
+
+ + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
group_assignmentnull
member_idThe member id assigned by the group coordinator or null if joining for the first time.
member_assignmentnull
+

+

SyncGroup Request (Version: 2) => group_id generation_id member_id [group_assignment] 
+  group_id => STRING
+  generation_id => INT32
+  member_id => STRING
+  group_assignment => member_id member_assignment 
+    member_id => STRING
+    member_assignment => BYTES
+
+ + + + + + + + + + + + + + +
FieldDescription
group_idThe unique group identifier
generation_idThe generation of the group.
member_idThe member id assigned by the group coordinator or null if joining for the first time.
group_assignmentnull
member_idThe member id assigned by the group coordinator or null if joining for the first time.
member_assignmentnull
+

+Responses:
+

SyncGroup Response (Version: 0) => error_code member_assignment 
+  error_code => INT16
+  member_assignment => BYTES
+
+ + + + + + +
FieldDescription
error_codeResponse error code
member_assignmentnull
+

+

SyncGroup Response (Version: 1) => throttle_time_ms error_code member_assignment 
+  throttle_time_ms => INT32
+  error_code => INT16
+  member_assignment => BYTES
+
+ + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
member_assignmentnull
+

+

SyncGroup Response (Version: 2) => throttle_time_ms error_code member_assignment 
+  throttle_time_ms => INT32
+  error_code => INT16
+  member_assignment => BYTES
+
+ + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
member_assignmentnull
+

+
DescribeGroups API (Key: 15):
+ +Requests:
+

DescribeGroups Request (Version: 0) => [group_ids] 
+  group_ids => STRING
+
+ + + + +
FieldDescription
group_idsList of groupIds to request metadata for (an empty groupId array will return empty group metadata).
+

+

DescribeGroups Request (Version: 1) => [group_ids] 
+  group_ids => STRING
+
+ + + + +
FieldDescription
group_idsList of groupIds to request metadata for (an empty groupId array will return empty group metadata).
+

+

DescribeGroups Request (Version: 2) => [group_ids] 
+  group_ids => STRING
+
+ + + + +
FieldDescription
group_idsList of groupIds to request metadata for (an empty groupId array will return empty group metadata).
+

+Responses:
+

DescribeGroups Response (Version: 0) => [groups] 
+  groups => error_code group_id state protocol_type protocol [members] 
+    error_code => INT16
+    group_id => STRING
+    state => STRING
+    protocol_type => STRING
+    protocol => STRING
+    members => member_id client_id client_host member_metadata member_assignment 
+      member_id => STRING
+      client_id => STRING
+      client_host => STRING
+      member_metadata => BYTES
+      member_assignment => BYTES
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
groupsnull
error_codeResponse error code
group_idThe unique group identifier
stateThe current state of the group (one of: Dead, Stable, CompletingRebalance, PreparingRebalance, or empty if there is no active group)
protocol_typeThe current group protocol type (will be empty if there is no active group)
protocolThe current group protocol (only provided if the group is Stable)
membersCurrent group members (only provided if the group is not Dead)
member_idThe member id assigned by the group coordinator or null if joining for the first time.
client_idThe client id used in the member's latest join group request
client_hostThe client host used in the request session corresponding to the member's join group.
member_metadataThe metadata corresponding to the current group protocol in use (will only be present if the group is stable).
member_assignmentThe current assignment provided by the group leader (will only be present if the group is stable).
+

+

DescribeGroups Response (Version: 1) => throttle_time_ms [groups] 
+  throttle_time_ms => INT32
+  groups => error_code group_id state protocol_type protocol [members] 
+    error_code => INT16
+    group_id => STRING
+    state => STRING
+    protocol_type => STRING
+    protocol => STRING
+    members => member_id client_id client_host member_metadata member_assignment 
+      member_id => STRING
+      client_id => STRING
+      client_host => STRING
+      member_metadata => BYTES
+      member_assignment => BYTES
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
groupsnull
error_codeResponse error code
group_idThe unique group identifier
stateThe current state of the group (one of: Dead, Stable, CompletingRebalance, PreparingRebalance, or empty if there is no active group)
protocol_typeThe current group protocol type (will be empty if there is no active group)
protocolThe current group protocol (only provided if the group is Stable)
membersCurrent group members (only provided if the group is not Dead)
member_idThe member id assigned by the group coordinator or null if joining for the first time.
client_idThe client id used in the member's latest join group request
client_hostThe client host used in the request session corresponding to the member's join group.
member_metadataThe metadata corresponding to the current group protocol in use (will only be present if the group is stable).
member_assignmentThe current assignment provided by the group leader (will only be present if the group is stable).
+

+

DescribeGroups Response (Version: 2) => throttle_time_ms [groups] 
+  throttle_time_ms => INT32
+  groups => error_code group_id state protocol_type protocol [members] 
+    error_code => INT16
+    group_id => STRING
+    state => STRING
+    protocol_type => STRING
+    protocol => STRING
+    members => member_id client_id client_host member_metadata member_assignment 
+      member_id => STRING
+      client_id => STRING
+      client_host => STRING
+      member_metadata => BYTES
+      member_assignment => BYTES
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
groupsnull
error_codeResponse error code
group_idThe unique group identifier
stateThe current state of the group (one of: Dead, Stable, CompletingRebalance, PreparingRebalance, or empty if there is no active group)
protocol_typeThe current group protocol type (will be empty if there is no active group)
protocolThe current group protocol (only provided if the group is Stable)
membersCurrent group members (only provided if the group is not Dead)
member_idThe member id assigned by the group coordinator or null if joining for the first time.
client_idThe client id used in the member's latest join group request
client_hostThe client host used in the request session corresponding to the member's join group.
member_metadataThe metadata corresponding to the current group protocol in use (will only be present if the group is stable).
member_assignmentThe current assignment provided by the group leader (will only be present if the group is stable).
+

+
ListGroups API (Key: 16):
+ +Requests:
+

ListGroups Request (Version: 0) => 
+
+ + +
FieldDescription
+

+

ListGroups Request (Version: 1) => 
+
+ + +
FieldDescription
+

+

ListGroups Request (Version: 2) => 
+
+ + +
FieldDescription
+

+Responses:
+

ListGroups Response (Version: 0) => error_code [groups] 
+  error_code => INT16
+  groups => group_id protocol_type 
+    group_id => STRING
+    protocol_type => STRING
+
+ + + + + + + + + + +
FieldDescription
error_codeResponse error code
groupsnull
group_idThe unique group identifier
protocol_typenull
+

+

ListGroups Response (Version: 1) => throttle_time_ms error_code [groups] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  groups => group_id protocol_type 
+    group_id => STRING
+    protocol_type => STRING
+
+ + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
groupsnull
group_idThe unique group identifier
protocol_typenull
+

+

ListGroups Response (Version: 2) => throttle_time_ms error_code [groups] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  groups => group_id protocol_type 
+    group_id => STRING
+    protocol_type => STRING
+
+ + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
groupsnull
group_idThe unique group identifier
protocol_typenull
+

+
SaslHandshake API (Key: 17):
+ +Requests:
+

SaslHandshake Request (Version: 0) => mechanism 
+  mechanism => STRING
+
+ + + + +
FieldDescription
mechanismSASL Mechanism chosen by the client.
+

+

SaslHandshake Request (Version: 1) => mechanism 
+  mechanism => STRING
+
+ + + + +
FieldDescription
mechanismSASL Mechanism chosen by the client.
+

+Responses:
+

SaslHandshake Response (Version: 0) => error_code [enabled_mechanisms] 
+  error_code => INT16
+  enabled_mechanisms => STRING
+
+ + + + + + +
FieldDescription
error_codeResponse error code
enabled_mechanismsArray of mechanisms enabled in the server.
+

+

SaslHandshake Response (Version: 1) => error_code [enabled_mechanisms] 
+  error_code => INT16
+  enabled_mechanisms => STRING
+
+ + + + + + +
FieldDescription
error_codeResponse error code
enabled_mechanismsArray of mechanisms enabled in the server.
+

+
ApiVersions API (Key: 18):
+ +Requests:
+

ApiVersions Request (Version: 0) => 
+
+ + +
FieldDescription
+

+

ApiVersions Request (Version: 1) => 
+
+ + +
FieldDescription
+

+

ApiVersions Request (Version: 2) => 
+
+ + +
FieldDescription
+

+Responses:
+

ApiVersions Response (Version: 0) => error_code [api_versions] 
+  error_code => INT16
+  api_versions => api_key min_version max_version 
+    api_key => INT16
+    min_version => INT16
+    max_version => INT16
+
+ + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
api_versionsAPI versions supported by the broker.
api_keyAPI key.
min_versionMinimum supported version.
max_versionMaximum supported version.
+

+

ApiVersions Response (Version: 1) => error_code [api_versions] throttle_time_ms 
+  error_code => INT16
+  api_versions => api_key min_version max_version 
+    api_key => INT16
+    min_version => INT16
+    max_version => INT16
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
api_versionsAPI versions supported by the broker.
api_keyAPI key.
min_versionMinimum supported version.
max_versionMaximum supported version.
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

ApiVersions Response (Version: 2) => error_code [api_versions] throttle_time_ms 
+  error_code => INT16
+  api_versions => api_key min_version max_version 
+    api_key => INT16
+    min_version => INT16
+    max_version => INT16
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
api_versionsAPI versions supported by the broker.
api_keyAPI key.
min_versionMinimum supported version.
max_versionMaximum supported version.
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+
CreateTopics API (Key: 19):
+ +Requests:
+

CreateTopics Request (Version: 0) => [create_topic_requests] timeout 
+  create_topic_requests => topic num_partitions replication_factor [replica_assignment] [config_entries] 
+    topic => STRING
+    num_partitions => INT32
+    replication_factor => INT16
+    replica_assignment => partition [replicas] 
+      partition => INT32
+      replicas => INT32
+    config_entries => config_name config_value 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+  timeout => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
create_topic_requestsAn array of single topic creation requests. Can not have multiple entries for the same topic.
topicName of topic
num_partitionsNumber of partitions to be created. -1 indicates unset.
replication_factorReplication factor for the topic. -1 indicates unset.
replica_assignmentReplica assignment among kafka brokers for this topic partitions. If this is set num_partitions and replication_factor must be unset.
partitionTopic partition id
replicasThe set of all nodes that should host this partition. The first replica in the list is the preferred leader.
config_entriesTopic level configuration for topic to be set.
config_nameConfiguration name
config_valueConfiguration value
timeoutThe time in ms to wait for a topic to be completely created on the controller node. Values <= 0 will trigger topic creation and return immediately
+

+

CreateTopics Request (Version: 1) => [create_topic_requests] timeout validate_only 
+  create_topic_requests => topic num_partitions replication_factor [replica_assignment] [config_entries] 
+    topic => STRING
+    num_partitions => INT32
+    replication_factor => INT16
+    replica_assignment => partition [replicas] 
+      partition => INT32
+      replicas => INT32
+    config_entries => config_name config_value 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+  timeout => INT32
+  validate_only => BOOLEAN
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
create_topic_requestsAn array of single topic creation requests. Can not have multiple entries for the same topic.
topicName of topic
num_partitionsNumber of partitions to be created. -1 indicates unset.
replication_factorReplication factor for the topic. -1 indicates unset.
replica_assignmentReplica assignment among kafka brokers for this topic partitions. If this is set num_partitions and replication_factor must be unset.
partitionTopic partition id
replicasThe set of all nodes that should host this partition. The first replica in the list is the preferred leader.
config_entriesTopic level configuration for topic to be set.
config_nameConfiguration name
config_valueConfiguration value
timeoutThe time in ms to wait for a topic to be completely created on the controller node. Values <= 0 will trigger topic creation and return immediately
validate_onlyIf this is true, the request will be validated, but the topic won't be created.
+

+

CreateTopics Request (Version: 2) => [create_topic_requests] timeout validate_only 
+  create_topic_requests => topic num_partitions replication_factor [replica_assignment] [config_entries] 
+    topic => STRING
+    num_partitions => INT32
+    replication_factor => INT16
+    replica_assignment => partition [replicas] 
+      partition => INT32
+      replicas => INT32
+    config_entries => config_name config_value 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+  timeout => INT32
+  validate_only => BOOLEAN
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
create_topic_requestsAn array of single topic creation requests. Can not have multiple entries for the same topic.
topicName of topic
num_partitionsNumber of partitions to be created. -1 indicates unset.
replication_factorReplication factor for the topic. -1 indicates unset.
replica_assignmentReplica assignment among kafka brokers for this topic partitions. If this is set num_partitions and replication_factor must be unset.
partitionTopic partition id
replicasThe set of all nodes that should host this partition. The first replica in the list is the preferred leader.
config_entriesTopic level configuration for topic to be set.
config_nameConfiguration name
config_valueConfiguration value
timeoutThe time in ms to wait for a topic to be completely created on the controller node. Values <= 0 will trigger topic creation and return immediately
validate_onlyIf this is true, the request will be validated, but the topic won't be created.
+

+

CreateTopics Request (Version: 3) => [create_topic_requests] timeout validate_only 
+  create_topic_requests => topic num_partitions replication_factor [replica_assignment] [config_entries] 
+    topic => STRING
+    num_partitions => INT32
+    replication_factor => INT16
+    replica_assignment => partition [replicas] 
+      partition => INT32
+      replicas => INT32
+    config_entries => config_name config_value 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+  timeout => INT32
+  validate_only => BOOLEAN
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
create_topic_requestsAn array of single topic creation requests. Can not have multiple entries for the same topic.
topicName of topic
num_partitionsNumber of partitions to be created. -1 indicates unset.
replication_factorReplication factor for the topic. -1 indicates unset.
replica_assignmentReplica assignment among kafka brokers for this topic partitions. If this is set num_partitions and replication_factor must be unset.
partitionTopic partition id
replicasThe set of all nodes that should host this partition. The first replica in the list is the preferred leader.
config_entriesTopic level configuration for topic to be set.
config_nameConfiguration name
config_valueConfiguration value
timeoutThe time in ms to wait for a topic to be completely created on the controller node. Values <= 0 will trigger topic creation and return immediately
validate_onlyIf this is true, the request will be validated, but the topic won't be created.
+

+Responses:
+

CreateTopics Response (Version: 0) => [topic_errors] 
+  topic_errors => topic error_code 
+    topic => STRING
+    error_code => INT16
+
+ + + + + + + + +
FieldDescription
topic_errorsAn array of per topic error codes.
topicName of topic
error_codeResponse error code
+

+

CreateTopics Response (Version: 1) => [topic_errors] 
+  topic_errors => topic error_code error_message 
+    topic => STRING
+    error_code => INT16
+    error_message => NULLABLE_STRING
+
+ + + + + + + + + + +
FieldDescription
topic_errorsAn array of per topic errors.
topicName of topic
error_codeResponse error code
error_messageResponse error message
+

+

CreateTopics Response (Version: 2) => throttle_time_ms [topic_errors] 
+  throttle_time_ms => INT32
+  topic_errors => topic error_code error_message 
+    topic => STRING
+    error_code => INT16
+    error_message => NULLABLE_STRING
+
+ + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topic_errorsAn array of per topic errors.
topicName of topic
error_codeResponse error code
error_messageResponse error message
+

+

CreateTopics Response (Version: 3) => throttle_time_ms [topic_errors] 
+  throttle_time_ms => INT32
+  topic_errors => topic error_code error_message 
+    topic => STRING
+    error_code => INT16
+    error_message => NULLABLE_STRING
+
+ + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topic_errorsAn array of per topic errors.
topicName of topic
error_codeResponse error code
error_messageResponse error message
+

+
DeleteTopics API (Key: 20):
+ +Requests:
+

DeleteTopics Request (Version: 0) => [topics] timeout 
+  topics => STRING
+  timeout => INT32
+
+ + + + + + +
FieldDescription
topicsAn array of topics to be deleted.
timeoutThe time in ms to wait for a topic to be completely deleted on the controller node. Values <= 0 will trigger topic deletion and return immediately
+

+

DeleteTopics Request (Version: 1) => [topics] timeout 
+  topics => STRING
+  timeout => INT32
+
+ + + + + + +
FieldDescription
topicsAn array of topics to be deleted.
timeoutThe time in ms to wait for a topic to be completely deleted on the controller node. Values <= 0 will trigger topic deletion and return immediately
+

+

DeleteTopics Request (Version: 2) => [topics] timeout 
+  topics => STRING
+  timeout => INT32
+
+ + + + + + +
FieldDescription
topicsAn array of topics to be deleted.
timeoutThe time in ms to wait for a topic to be completely deleted on the controller node. Values <= 0 will trigger topic deletion and return immediately
+

+

DeleteTopics Request (Version: 3) => [topics] timeout 
+  topics => STRING
+  timeout => INT32
+
+ + + + + + +
FieldDescription
topicsAn array of topics to be deleted.
timeoutThe time in ms to wait for a topic to be completely deleted on the controller node. Values <= 0 will trigger topic deletion and return immediately
+

+Responses:
+

DeleteTopics Response (Version: 0) => [topic_error_codes] 
+  topic_error_codes => topic error_code 
+    topic => STRING
+    error_code => INT16
+
+ + + + + + + + +
FieldDescription
topic_error_codesAn array of per topic error codes.
topicName of topic
error_codeResponse error code
+

+

DeleteTopics Response (Version: 1) => throttle_time_ms [topic_error_codes] 
+  throttle_time_ms => INT32
+  topic_error_codes => topic error_code 
+    topic => STRING
+    error_code => INT16
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topic_error_codesAn array of per topic error codes.
topicName of topic
error_codeResponse error code
+

+

DeleteTopics Response (Version: 2) => throttle_time_ms [topic_error_codes] 
+  throttle_time_ms => INT32
+  topic_error_codes => topic error_code 
+    topic => STRING
+    error_code => INT16
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topic_error_codesAn array of per topic error codes.
topicName of topic
error_codeResponse error code
+

+

DeleteTopics Response (Version: 3) => throttle_time_ms [topic_error_codes] 
+  throttle_time_ms => INT32
+  topic_error_codes => topic error_code 
+    topic => STRING
+    error_code => INT16
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topic_error_codesAn array of per topic error codes.
topicName of topic
error_codeResponse error code
+

+
DeleteRecords API (Key: 21):
+ +Requests:
+

DeleteRecords Request (Version: 0) => [topics] timeout 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset 
+      partition => INT32
+      offset => INT64
+  timeout => INT32
+
+ + + + + + + + + + + + + + +
FieldDescription
topicsnull
topicName of topic
partitionsnull
partitionTopic partition id
offsetThe offset before which the messages will be deleted. -1 means high-watermark for the partition.
timeoutThe maximum time to await a response in ms.
+

+

DeleteRecords Request (Version: 1) => [topics] timeout 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset 
+      partition => INT32
+      offset => INT64
+  timeout => INT32
+
+ + + + + + + + + + + + + + +
FieldDescription
topicsnull
topicName of topic
partitionsnull
partitionTopic partition id
offsetThe offset before which the messages will be deleted. -1 means high-watermark for the partition.
timeoutThe maximum time to await a response in ms.
+

+Responses:
+

DeleteRecords Response (Version: 0) => throttle_time_ms [topics] 
+  throttle_time_ms => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition low_watermark error_code 
+      partition => INT32
+      low_watermark => INT64
+      error_code => INT16
+
+ + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topicsnull
topicName of topic
partitionsnull
partitionTopic partition id
low_watermarkSmallest available offset of all live replicas
error_codeResponse error code
+

+

DeleteRecords Response (Version: 1) => throttle_time_ms [topics] 
+  throttle_time_ms => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition low_watermark error_code 
+      partition => INT32
+      low_watermark => INT64
+      error_code => INT16
+
+ + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topicsnull
topicName of topic
partitionsnull
partitionTopic partition id
low_watermarkSmallest available offset of all live replicas
error_codeResponse error code
+

+
InitProducerId API (Key: 22):
+ +Requests:
+

InitProducerId Request (Version: 0) => transactional_id transaction_timeout_ms 
+  transactional_id => NULLABLE_STRING
+  transaction_timeout_ms => INT32
+
+ + + + + + +
FieldDescription
transactional_idThe transactional id or null if the producer is not transactional
transaction_timeout_msThe time in ms to wait for before aborting idle transactions sent by this producer.
+

+

InitProducerId Request (Version: 1) => transactional_id transaction_timeout_ms 
+  transactional_id => NULLABLE_STRING
+  transaction_timeout_ms => INT32
+
+ + + + + + +
FieldDescription
transactional_idThe transactional id or null if the producer is not transactional
transaction_timeout_msThe time in ms to wait for before aborting idle transactions sent by this producer.
+

+Responses:
+

InitProducerId Response (Version: 0) => throttle_time_ms error_code producer_id producer_epoch 
+  throttle_time_ms => INT32
+  error_code => INT16
+  producer_id => INT64
+  producer_epoch => INT16
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
+

+

InitProducerId Response (Version: 1) => throttle_time_ms error_code producer_id producer_epoch 
+  throttle_time_ms => INT32
+  error_code => INT16
+  producer_id => INT64
+  producer_epoch => INT16
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
+

+
OffsetForLeaderEpoch API (Key: 23):
+ +Requests:
+

OffsetForLeaderEpoch Request (Version: 0) => [topics] 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition leader_epoch 
+      partition => INT32
+      leader_epoch => INT32
+
+ + + + + + + + + + + + +
FieldDescription
topicsAn array of topics to get epochs for
topicName of topic
partitionsAn array of partitions to get epochs for
partitionTopic partition id
leader_epochThe epoch to lookup an offset for.
+

+

OffsetForLeaderEpoch Request (Version: 1) => [topics] 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition leader_epoch 
+      partition => INT32
+      leader_epoch => INT32
+
+ + + + + + + + + + + + +
FieldDescription
topicsAn array of topics to get epochs for
topicName of topic
partitionsAn array of partitions to get epochs for
partitionTopic partition id
leader_epochThe epoch to lookup an offset for.
+

+

OffsetForLeaderEpoch Request (Version: 2) => [topics] 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition current_leader_epoch leader_epoch 
+      partition => INT32
+      current_leader_epoch => INT32
+      leader_epoch => INT32
+
+ + + + + + + + + + + + + + +
FieldDescription
topicsAn array of topics to get epochs for
topicName of topic
partitionsAn array of partitions to get epochs for
partitionTopic partition id
current_leader_epochThe current leader epoch, if provided, is used to fence consumers/replicas with old metadata. If the epoch provided by the client is larger than the current epoch known to the broker, then the UNKNOWN_LEADER_EPOCH error code will be returned. If the provided epoch is smaller, then the FENCED_LEADER_EPOCH error code will be returned.
leader_epochThe epoch to lookup an offset for.
+

+Responses:
+

OffsetForLeaderEpoch Response (Version: 0) => [topics] 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => error_code partition end_offset 
+      error_code => INT16
+      partition => INT32
+      end_offset => INT64
+
+ + + + + + + + + + + + + + +
FieldDescription
topicsAn array of topics for which we have leader offsets for some requested partition leader epoch
topicName of topic
partitionsAn array of offsets by partition
error_codeResponse error code
partitionTopic partition id
end_offsetThe end offset
+

+

OffsetForLeaderEpoch Response (Version: 1) => [topics] 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => error_code partition leader_epoch end_offset 
+      error_code => INT16
+      partition => INT32
+      leader_epoch => INT32
+      end_offset => INT64
+
+ + + + + + + + + + + + + + + + +
FieldDescription
topicsAn array of topics for which we have leader offsets for some requested partition leader epoch
topicName of topic
partitionsAn array of offsets by partition
error_codeResponse error code
partitionTopic partition id
leader_epochThe leader epoch
end_offsetThe end offset
+

+

OffsetForLeaderEpoch Response (Version: 2) => throttle_time_ms [topics] 
+  throttle_time_ms => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => error_code partition leader_epoch end_offset 
+      error_code => INT16
+      partition => INT32
+      leader_epoch => INT32
+      end_offset => INT64
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topicsAn array of topics for which we have leader offsets for some requested partition leader epoch
topicName of topic
partitionsAn array of offsets by partition
error_codeResponse error code
partitionTopic partition id
leader_epochThe leader epoch
end_offsetThe end offset
+

+
AddPartitionsToTxn API (Key: 24):
+ +Requests:
+

AddPartitionsToTxn Request (Version: 0) => transactional_id producer_id producer_epoch [topics] 
+  transactional_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => INT32
+
+ + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
topicsThe partitions to add to the transaction.
topicName of topic
partitionsnull
+

+

AddPartitionsToTxn Request (Version: 1) => transactional_id producer_id producer_epoch [topics] 
+  transactional_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => INT32
+
+ + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
topicsThe partitions to add to the transaction.
topicName of topic
partitionsnull
+

+Responses:
+

AddPartitionsToTxn Response (Version: 0) => throttle_time_ms [errors] 
+  throttle_time_ms => INT32
+  errors => topic [partition_errors] 
+    topic => STRING
+    partition_errors => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
errorsnull
topicName of topic
partition_errorsnull
partitionTopic partition id
error_codeResponse error code
+

+

AddPartitionsToTxn Response (Version: 1) => throttle_time_ms [errors] 
+  throttle_time_ms => INT32
+  errors => topic [partition_errors] 
+    topic => STRING
+    partition_errors => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
errorsnull
topicName of topic
partition_errorsnull
partitionTopic partition id
error_codeResponse error code
+

+
AddOffsetsToTxn API (Key: 25):
+ +Requests:
+

AddOffsetsToTxn Request (Version: 0) => transactional_id producer_id producer_epoch group_id 
+  transactional_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  group_id => STRING
+
+ + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
group_idThe unique group identifier
+

+

AddOffsetsToTxn Request (Version: 1) => transactional_id producer_id producer_epoch group_id 
+  transactional_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  group_id => STRING
+
+ + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
group_idThe unique group identifier
+

+Responses:
+

AddOffsetsToTxn Response (Version: 0) => throttle_time_ms error_code 
+  throttle_time_ms => INT32
+  error_code => INT16
+
+ + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
+

+

AddOffsetsToTxn Response (Version: 1) => throttle_time_ms error_code 
+  throttle_time_ms => INT32
+  error_code => INT16
+
+ + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
+

+
EndTxn API (Key: 26):
+ +Requests:
+

EndTxn Request (Version: 0) => transactional_id producer_id producer_epoch transaction_result 
+  transactional_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  transaction_result => BOOLEAN
+
+ + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
transaction_resultThe result of the transaction (0 = ABORT, 1 = COMMIT)
+

+

EndTxn Request (Version: 1) => transactional_id producer_id producer_epoch transaction_result 
+  transactional_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  transaction_result => BOOLEAN
+
+ + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
transaction_resultThe result of the transaction (0 = ABORT, 1 = COMMIT)
+

+Responses:
+

EndTxn Response (Version: 0) => throttle_time_ms error_code 
+  throttle_time_ms => INT32
+  error_code => INT16
+
+ + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
+

+

EndTxn Response (Version: 1) => throttle_time_ms error_code 
+  throttle_time_ms => INT32
+  error_code => INT16
+
+ + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
+

+
WriteTxnMarkers API (Key: 27):
+ +Requests:
+

WriteTxnMarkers Request (Version: 0) => [transaction_markers] 
+  transaction_markers => producer_id producer_epoch transaction_result [topics] coordinator_epoch 
+    producer_id => INT64
+    producer_epoch => INT16
+    transaction_result => BOOLEAN
+    topics => topic [partitions] 
+      topic => STRING
+      partitions => INT32
+    coordinator_epoch => INT32
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
transaction_markersThe transaction markers to be written.
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
transaction_resultThe result of the transaction to write to the partitions (false = ABORT, true = COMMIT).
topicsThe partitions to write markers for.
topicName of topic
partitionsnull
coordinator_epochEpoch associated with the transaction state partition hosted by this transaction coordinator
+

+Responses:
+

WriteTxnMarkers Response (Version: 0) => [transaction_markers] 
+  transaction_markers => producer_id [topics] 
+    producer_id => INT64
+    topics => topic [partitions] 
+      topic => STRING
+      partitions => partition error_code 
+        partition => INT32
+        error_code => INT16
+
+ + + + + + + + + + + + + + + + +
FieldDescription
transaction_markersErrors per partition from writing markers.
producer_idCurrent producer id in use by the transactional id.
topicsErrors per partition from writing markers.
topicName of topic
partitionsnull
partitionTopic partition id
error_codeResponse error code
+

+
TxnOffsetCommit API (Key: 28):
+ +Requests:
+

TxnOffsetCommit Request (Version: 0) => transactional_id group_id producer_id producer_epoch [topics] 
+  transactional_id => STRING
+  group_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset metadata 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
group_idThe unique group identifier
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
+

+

TxnOffsetCommit Request (Version: 1) => transactional_id group_id producer_id producer_epoch [topics] 
+  transactional_id => STRING
+  group_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset metadata 
+      partition => INT32
+      offset => INT64
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
group_idThe unique group identifier
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
metadataAny associated metadata the client wants to keep.
+

+

TxnOffsetCommit Request (Version: 2) => transactional_id group_id producer_id producer_epoch [topics] 
+  transactional_id => STRING
+  group_id => STRING
+  producer_id => INT64
+  producer_epoch => INT16
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition offset leader_epoch metadata 
+      partition => INT32
+      offset => INT64
+      leader_epoch => INT32
+      metadata => NULLABLE_STRING
+
+ + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
transactional_idThe transactional id corresponding to the transaction.
group_idThe unique group identifier
producer_idCurrent producer id in use by the transactional id.
producer_epochCurrent epoch associated with the producer id.
topicsTopics to commit offsets
topicName of topic
partitionsPartitions to commit offsets
partitionTopic partition id
offsetMessage offset to be committed
leader_epochThe leader epoch, if provided is derived from the last consumed record. This is used by the consumer to check for log truncation and to ensure partition metadata is up to date following a group rebalance.
metadataAny associated metadata the client wants to keep.
+

+Responses:
+

TxnOffsetCommit Response (Version: 0) => throttle_time_ms [topics] 
+  throttle_time_ms => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topicsResponses by topic for committed offsets
topicName of topic
partitionsResponses by partition for committed offsets
partitionTopic partition id
error_codeResponse error code
+

+

TxnOffsetCommit Response (Version: 1) => throttle_time_ms [topics] 
+  throttle_time_ms => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topicsResponses by topic for committed offsets
topicName of topic
partitionsResponses by partition for committed offsets
partitionTopic partition id
error_codeResponse error code
+

+

TxnOffsetCommit Response (Version: 2) => throttle_time_ms [topics] 
+  throttle_time_ms => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topicsResponses by topic for committed offsets
topicName of topic
partitionsResponses by partition for committed offsets
partitionTopic partition id
error_codeResponse error code
+

+
DescribeAcls API (Key: 29):
+ +Requests:
+

DescribeAcls Request (Version: 0) => resource_type resource_name principal host operation permission_type 
+  resource_type => INT8
+  resource_name => NULLABLE_STRING
+  principal => NULLABLE_STRING
+  host => NULLABLE_STRING
+  operation => INT8
+  permission_type => INT8
+
+ + + + + + + + + + + + + + +
FieldDescription
resource_typeThe resource type
resource_nameThe resource name filter
principalThe ACL principal filter
hostThe ACL host filter
operationThe ACL operation
permission_typeThe ACL permission type
+

+

DescribeAcls Request (Version: 1) => resource_type resource_name resource_pattern_type_filter principal host operation permission_type 
+  resource_type => INT8
+  resource_name => NULLABLE_STRING
+  resource_pattern_type_filter => INT8
+  principal => NULLABLE_STRING
+  host => NULLABLE_STRING
+  operation => INT8
+  permission_type => INT8
+
+ + + + + + + + + + + + + + + + +
FieldDescription
resource_typeThe resource type
resource_nameThe resource name filter
resource_pattern_type_filterThe resource pattern type filter
principalThe ACL principal filter
hostThe ACL host filter
operationThe ACL operation
permission_typeThe ACL permission type
+

+Responses:
+

DescribeAcls Response (Version: 0) => throttle_time_ms error_code error_message [resources] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  error_message => NULLABLE_STRING
+  resources => resource_type resource_name [acls] 
+    resource_type => INT8
+    resource_name => STRING
+    acls => principal host operation permission_type 
+      principal => STRING
+      host => STRING
+      operation => INT8
+      permission_type => INT8
+
+ + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
error_messageResponse error message
resourcesThe resources and their associated ACLs.
resource_typeThe resource type
resource_nameThe resource name
aclsnull
principalThe ACL principal
hostThe ACL host
operationThe ACL operation
permission_typeThe ACL permission type
+

+

DescribeAcls Response (Version: 1) => throttle_time_ms error_code error_message [resources] 
+  throttle_time_ms => INT32
+  error_code => INT16
+  error_message => NULLABLE_STRING
+  resources => resource_type resource_name resource_pattten_type [acls] 
+    resource_type => INT8
+    resource_name => STRING
+    resource_pattten_type => INT8
+    acls => principal host operation permission_type 
+      principal => STRING
+      host => STRING
+      operation => INT8
+      permission_type => INT8
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
error_codeResponse error code
error_messageResponse error message
resourcesThe resources and their associated ACLs.
resource_typeThe resource type
resource_nameThe resource name
resource_pattten_typeThe resource pattern type
aclsnull
principalThe ACL principal
hostThe ACL host
operationThe ACL operation
permission_typeThe ACL permission type
+

+
CreateAcls API (Key: 30):
+ +Requests:
+

CreateAcls Request (Version: 0) => [creations] 
+  creations => resource_type resource_name principal host operation permission_type 
+    resource_type => INT8
+    resource_name => STRING
+    principal => STRING
+    host => STRING
+    operation => INT8
+    permission_type => INT8
+
+ + + + + + + + + + + + + + + + +
FieldDescription
creationsnull
resource_typeThe resource type
resource_nameThe resource name
principalThe ACL principal
hostThe ACL host
operationThe ACL operation
permission_typeThe ACL permission type
+

+

CreateAcls Request (Version: 1) => [creations] 
+  creations => resource_type resource_name resource_pattten_type principal host operation permission_type 
+    resource_type => INT8
+    resource_name => STRING
+    resource_pattten_type => INT8
+    principal => STRING
+    host => STRING
+    operation => INT8
+    permission_type => INT8
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
creationsnull
resource_typeThe resource type
resource_nameThe resource name
resource_pattten_typeThe resource pattern type
principalThe ACL principal
hostThe ACL host
operationThe ACL operation
permission_typeThe ACL permission type
+

+Responses:
+

CreateAcls Response (Version: 0) => throttle_time_ms [creation_responses] 
+  throttle_time_ms => INT32
+  creation_responses => error_code error_message 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
creation_responsesnull
error_codeResponse error code
error_messageResponse error message
+

+

CreateAcls Response (Version: 1) => throttle_time_ms [creation_responses] 
+  throttle_time_ms => INT32
+  creation_responses => error_code error_message 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
creation_responsesnull
error_codeResponse error code
error_messageResponse error message
+

+
DeleteAcls API (Key: 31):
+ +Requests:
+

DeleteAcls Request (Version: 0) => [filters] 
+  filters => resource_type resource_name principal host operation permission_type 
+    resource_type => INT8
+    resource_name => NULLABLE_STRING
+    principal => NULLABLE_STRING
+    host => NULLABLE_STRING
+    operation => INT8
+    permission_type => INT8
+
+ + + + + + + + + + + + + + + + +
FieldDescription
filtersnull
resource_typeThe resource type
resource_nameThe resource name filter
principalThe ACL principal filter
hostThe ACL host filter
operationThe ACL operation
permission_typeThe ACL permission type
+

+

DeleteAcls Request (Version: 1) => [filters] 
+  filters => resource_type resource_name resource_pattern_type_filter principal host operation permission_type 
+    resource_type => INT8
+    resource_name => NULLABLE_STRING
+    resource_pattern_type_filter => INT8
+    principal => NULLABLE_STRING
+    host => NULLABLE_STRING
+    operation => INT8
+    permission_type => INT8
+
+ + + + + + + + + + + + + + + + + + +
FieldDescription
filtersnull
resource_typeThe resource type
resource_nameThe resource name filter
resource_pattern_type_filterThe resource pattern type filter
principalThe ACL principal filter
hostThe ACL host filter
operationThe ACL operation
permission_typeThe ACL permission type
+

+Responses:
+

DeleteAcls Response (Version: 0) => throttle_time_ms [filter_responses] 
+  throttle_time_ms => INT32
+  filter_responses => error_code error_message [matching_acls] 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+    matching_acls => error_code error_message resource_type resource_name principal host operation permission_type 
+      error_code => INT16
+      error_message => NULLABLE_STRING
+      resource_type => INT8
+      resource_name => STRING
+      principal => STRING
+      host => STRING
+      operation => INT8
+      permission_type => INT8
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
filter_responsesnull
error_codeResponse error code
error_messageResponse error message
matching_aclsThe matching ACLs
error_codeResponse error code
error_messageResponse error message
resource_typeThe resource type
resource_nameThe resource name
principalThe ACL principal
hostThe ACL host
operationThe ACL operation
permission_typeThe ACL permission type
+

+

DeleteAcls Response (Version: 1) => throttle_time_ms [filter_responses] 
+  throttle_time_ms => INT32
+  filter_responses => error_code error_message [matching_acls] 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+    matching_acls => error_code error_message resource_type resource_name resource_pattten_type principal host operation permission_type 
+      error_code => INT16
+      error_message => NULLABLE_STRING
+      resource_type => INT8
+      resource_name => STRING
+      resource_pattten_type => INT8
+      principal => STRING
+      host => STRING
+      operation => INT8
+      permission_type => INT8
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
filter_responsesnull
error_codeResponse error code
error_messageResponse error message
matching_aclsThe matching ACLs
error_codeResponse error code
error_messageResponse error message
resource_typeThe resource type
resource_nameThe resource name
resource_pattten_typeThe resource pattern type
principalThe ACL principal
hostThe ACL host
operationThe ACL operation
permission_typeThe ACL permission type
+

+
DescribeConfigs API (Key: 32):
+ +Requests:
+

DescribeConfigs Request (Version: 0) => [resources] 
+  resources => resource_type resource_name [config_names] 
+    resource_type => INT8
+    resource_name => STRING
+    config_names => STRING
+
+ + + + + + + + + + +
FieldDescription
resourcesAn array of config resources to be returned.
resource_typenull
resource_namenull
config_namesnull
+

+

DescribeConfigs Request (Version: 1) => [resources] include_synonyms 
+  resources => resource_type resource_name [config_names] 
+    resource_type => INT8
+    resource_name => STRING
+    config_names => STRING
+  include_synonyms => BOOLEAN
+
+ + + + + + + + + + + + +
FieldDescription
resourcesAn array of config resources to be returned.
resource_typenull
resource_namenull
config_namesnull
include_synonymsnull
+

+

DescribeConfigs Request (Version: 2) => [resources] include_synonyms 
+  resources => resource_type resource_name [config_names] 
+    resource_type => INT8
+    resource_name => STRING
+    config_names => STRING
+  include_synonyms => BOOLEAN
+
+ + + + + + + + + + + + +
FieldDescription
resourcesAn array of config resources to be returned.
resource_typenull
resource_namenull
config_namesnull
include_synonymsnull
+

+Responses:
+

DescribeConfigs Response (Version: 0) => throttle_time_ms [resources] 
+  throttle_time_ms => INT32
+  resources => error_code error_message resource_type resource_name [config_entries] 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+    resource_type => INT8
+    resource_name => STRING
+    config_entries => config_name config_value read_only is_default is_sensitive 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+      read_only => BOOLEAN
+      is_default => BOOLEAN
+      is_sensitive => BOOLEAN
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
resourcesnull
error_codeResponse error code
error_messageResponse error message
resource_typenull
resource_namenull
config_entriesnull
config_namenull
config_valuenull
read_onlynull
is_defaultnull
is_sensitivenull
+

+

DescribeConfigs Response (Version: 1) => throttle_time_ms [resources] 
+  throttle_time_ms => INT32
+  resources => error_code error_message resource_type resource_name [config_entries] 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+    resource_type => INT8
+    resource_name => STRING
+    config_entries => config_name config_value read_only config_source is_sensitive [config_synonyms] 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+      read_only => BOOLEAN
+      config_source => INT8
+      is_sensitive => BOOLEAN
+      config_synonyms => config_name config_value config_source 
+        config_name => STRING
+        config_value => NULLABLE_STRING
+        config_source => INT8
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
resourcesnull
error_codeResponse error code
error_messageResponse error message
resource_typenull
resource_namenull
config_entriesnull
config_namenull
config_valuenull
read_onlynull
config_sourcenull
is_sensitivenull
config_synonymsnull
config_namenull
config_valuenull
config_sourcenull
+

+

DescribeConfigs Response (Version: 2) => throttle_time_ms [resources] 
+  throttle_time_ms => INT32
+  resources => error_code error_message resource_type resource_name [config_entries] 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+    resource_type => INT8
+    resource_name => STRING
+    config_entries => config_name config_value read_only config_source is_sensitive [config_synonyms] 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+      read_only => BOOLEAN
+      config_source => INT8
+      is_sensitive => BOOLEAN
+      config_synonyms => config_name config_value config_source 
+        config_name => STRING
+        config_value => NULLABLE_STRING
+        config_source => INT8
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
resourcesnull
error_codeResponse error code
error_messageResponse error message
resource_typenull
resource_namenull
config_entriesnull
config_namenull
config_valuenull
read_onlynull
config_sourcenull
is_sensitivenull
config_synonymsnull
config_namenull
config_valuenull
config_sourcenull
+

+
AlterConfigs API (Key: 33):
+ +Requests:
+

AlterConfigs Request (Version: 0) => [resources] validate_only 
+  resources => resource_type resource_name [config_entries] 
+    resource_type => INT8
+    resource_name => STRING
+    config_entries => config_name config_value 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+  validate_only => BOOLEAN
+
+ + + + + + + + + + + + + + + + +
FieldDescription
resourcesAn array of resources to update with the provided configs.
resource_typenull
resource_namenull
config_entriesnull
config_nameConfiguration name
config_valueConfiguration value
validate_onlynull
+

+

AlterConfigs Request (Version: 1) => [resources] validate_only 
+  resources => resource_type resource_name [config_entries] 
+    resource_type => INT8
+    resource_name => STRING
+    config_entries => config_name config_value 
+      config_name => STRING
+      config_value => NULLABLE_STRING
+  validate_only => BOOLEAN
+
+ + + + + + + + + + + + + + + + +
FieldDescription
resourcesAn array of resources to update with the provided configs.
resource_typenull
resource_namenull
config_entriesnull
config_nameConfiguration name
config_valueConfiguration value
validate_onlynull
+

+Responses:
+

AlterConfigs Response (Version: 0) => throttle_time_ms [resources] 
+  throttle_time_ms => INT32
+  resources => error_code error_message resource_type resource_name 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+    resource_type => INT8
+    resource_name => STRING
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
resourcesnull
error_codeResponse error code
error_messageResponse error message
resource_typenull
resource_namenull
+

+

AlterConfigs Response (Version: 1) => throttle_time_ms [resources] 
+  throttle_time_ms => INT32
+  resources => error_code error_message resource_type resource_name 
+    error_code => INT16
+    error_message => NULLABLE_STRING
+    resource_type => INT8
+    resource_name => STRING
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
resourcesnull
error_codeResponse error code
error_messageResponse error message
resource_typenull
resource_namenull
+

+
AlterReplicaLogDirs API (Key: 34):
+ +Requests:
+

AlterReplicaLogDirs Request (Version: 0) => [log_dirs] 
+  log_dirs => log_dir [topics] 
+    log_dir => STRING
+    topics => topic [partitions] 
+      topic => STRING
+      partitions => INT32
+
+ + + + + + + + + + + + +
FieldDescription
log_dirsnull
log_dirThe absolute log directory path.
topicsnull
topicName of topic
partitionsList of partition ids of the topic.
+

+

AlterReplicaLogDirs Request (Version: 1) => [log_dirs] 
+  log_dirs => log_dir [topics] 
+    log_dir => STRING
+    topics => topic [partitions] 
+      topic => STRING
+      partitions => INT32
+
+ + + + + + + + + + + + +
FieldDescription
log_dirsnull
log_dirThe absolute log directory path.
topicsnull
topicName of topic
partitionsList of partition ids of the topic.
+

+Responses:
+

AlterReplicaLogDirs Response (Version: 0) => throttle_time_ms [topics] 
+  throttle_time_ms => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topicsnull
topicName of topic
partitionsnull
partitionTopic partition id
error_codeResponse error code
+

+

AlterReplicaLogDirs Response (Version: 1) => throttle_time_ms [topics] 
+  throttle_time_ms => INT32
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => partition error_code 
+      partition => INT32
+      error_code => INT16
+
+ + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topicsnull
topicName of topic
partitionsnull
partitionTopic partition id
error_codeResponse error code
+

+
DescribeLogDirs API (Key: 35):
+ +Requests:
+

DescribeLogDirs Request (Version: 0) => [topics] 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => INT32
+
+ + + + + + + + +
FieldDescription
topicsnull
topicName of topic
partitionsList of partition ids of the topic.
+

+

DescribeLogDirs Request (Version: 1) => [topics] 
+  topics => topic [partitions] 
+    topic => STRING
+    partitions => INT32
+
+ + + + + + + + +
FieldDescription
topicsnull
topicName of topic
partitionsList of partition ids of the topic.
+

+Responses:
+

DescribeLogDirs Response (Version: 0) => throttle_time_ms [log_dirs] 
+  throttle_time_ms => INT32
+  log_dirs => error_code log_dir [topics] 
+    error_code => INT16
+    log_dir => STRING
+    topics => topic [partitions] 
+      topic => STRING
+      partitions => partition size offset_lag is_future 
+        partition => INT32
+        size => INT64
+        offset_lag => INT64
+        is_future => BOOLEAN
+
+ + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
log_dirsnull
error_codeResponse error code
log_dirThe absolute log directory path.
topicsnull
topicName of topic
partitionsnull
partitionTopic partition id
sizeThe size of the log segments of the partition in bytes.
offset_lagThe lag of the log's LEO w.r.t. partition's HW (if it is the current log for the partition) or current replica's LEO (if it is the future log for the partition)
is_futureTrue if this log is created by AlterReplicaLogDirsRequest and will replace the current log of the replica in the future.
+

+

DescribeLogDirs Response (Version: 1) => throttle_time_ms [log_dirs] 
+  throttle_time_ms => INT32
+  log_dirs => error_code log_dir [topics] 
+    error_code => INT16
+    log_dir => STRING
+    topics => topic [partitions] 
+      topic => STRING
+      partitions => partition size offset_lag is_future 
+        partition => INT32
+        size => INT64
+        offset_lag => INT64
+        is_future => BOOLEAN
+
+ + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
log_dirsnull
error_codeResponse error code
log_dirThe absolute log directory path.
topicsnull
topicName of topic
partitionsnull
partitionTopic partition id
sizeThe size of the log segments of the partition in bytes.
offset_lagThe lag of the log's LEO w.r.t. partition's HW (if it is the current log for the partition) or current replica's LEO (if it is the future log for the partition)
is_futureTrue if this log is created by AlterReplicaLogDirsRequest and will replace the current log of the replica in the future.
+

+
SaslAuthenticate API (Key: 36):
+ +Requests:
+

SaslAuthenticate Request (Version: 0) => sasl_auth_bytes 
+  sasl_auth_bytes => BYTES
+
+ + + + +
FieldDescription
sasl_auth_bytesSASL authentication bytes from client as defined by the SASL mechanism.
+

+Responses:
+

SaslAuthenticate Response (Version: 0) => error_code error_message sasl_auth_bytes 
+  error_code => INT16
+  error_message => NULLABLE_STRING
+  sasl_auth_bytes => BYTES
+
+ + + + + + + + +
FieldDescription
error_codeResponse error code
error_messageResponse error message
sasl_auth_bytesSASL authentication bytes from server as defined by the SASL mechanism.
+

+
CreatePartitions API (Key: 37):
+ +Requests:
+

CreatePartitions Request (Version: 0) => [topic_partitions] timeout validate_only 
+  topic_partitions => topic new_partitions 
+    topic => STRING
+    new_partitions => count [assignment] 
+      count => INT32
+      assignment => ARRAY(INT32)
+  timeout => INT32
+  validate_only => BOOLEAN
+
+ + + + + + + + + + + + + + + + +
FieldDescription
topic_partitionsList of topic and the corresponding new partitions.
topicName of topic
new_partitionsnull
countThe new partition count.
assignmentThe assigned brokers.
timeoutThe time in ms to wait for the partitions to be created.
validate_onlyIf true then validate the request, but don't actually increase the number of partitions.
+

+

CreatePartitions Request (Version: 1) => [topic_partitions] timeout validate_only 
+  topic_partitions => topic new_partitions 
+    topic => STRING
+    new_partitions => count [assignment] 
+      count => INT32
+      assignment => ARRAY(INT32)
+  timeout => INT32
+  validate_only => BOOLEAN
+
+ + + + + + + + + + + + + + + + +
FieldDescription
topic_partitionsList of topic and the corresponding new partitions.
topicName of topic
new_partitionsnull
countThe new partition count.
assignmentThe assigned brokers.
timeoutThe time in ms to wait for the partitions to be created.
validate_onlyIf true then validate the request, but don't actually increase the number of partitions.
+

+Responses:
+

CreatePartitions Response (Version: 0) => throttle_time_ms [topic_errors] 
+  throttle_time_ms => INT32
+  topic_errors => topic error_code error_message 
+    topic => STRING
+    error_code => INT16
+    error_message => NULLABLE_STRING
+
+ + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topic_errorsPer topic results for the create partitions request
topicName of topic
error_codeResponse error code
error_messageResponse error message
+

+

CreatePartitions Response (Version: 1) => throttle_time_ms [topic_errors] 
+  throttle_time_ms => INT32
+  topic_errors => topic error_code error_message 
+    topic => STRING
+    error_code => INT16
+    error_message => NULLABLE_STRING
+
+ + + + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
topic_errorsPer topic results for the create partitions request
topicName of topic
error_codeResponse error code
error_messageResponse error message
+

+
CreateDelegationToken API (Key: 38):
+ +Requests:
+

CreateDelegationToken Request (Version: 0) => [renewers] max_life_time 
+  renewers => principal_type name 
+    principal_type => STRING
+    name => STRING
+  max_life_time => INT64
+
+ + + + + + + + + + +
FieldDescription
renewersAn array of token renewers. Renewer is an Kafka PrincipalType and name string, who is allowed to renew this token before the max lifetime expires.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
max_life_timeMax lifetime period for token in milli seconds. if value is -1, then max lifetime will default to a server side config value.
+

+

CreateDelegationToken Request (Version: 1) => [renewers] max_life_time 
+  renewers => principal_type name 
+    principal_type => STRING
+    name => STRING
+  max_life_time => INT64
+
+ + + + + + + + + + +
FieldDescription
renewersAn array of token renewers. Renewer is an Kafka PrincipalType and name string, who is allowed to renew this token before the max lifetime expires.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
max_life_timeMax lifetime period for token in milli seconds. if value is -1, then max lifetime will default to a server side config value.
+

+Responses:
+

CreateDelegationToken Response (Version: 0) => error_code owner issue_timestamp expiry_timestamp max_timestamp token_id hmac throttle_time_ms 
+  error_code => INT16
+  owner => principal_type name 
+    principal_type => STRING
+    name => STRING
+  issue_timestamp => INT64
+  expiry_timestamp => INT64
+  max_timestamp => INT64
+  token_id => STRING
+  hmac => BYTES
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
ownertoken owner.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
issue_timestamptimestamp (in msec) when this token was generated.
expiry_timestamptimestamp (in msec) at which this token expires.
max_timestampmax life time of this token.
token_idUUID to ensure uniqueness.
hmacHMAC of the delegation token.
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

CreateDelegationToken Response (Version: 1) => error_code owner issue_timestamp expiry_timestamp max_timestamp token_id hmac throttle_time_ms 
+  error_code => INT16
+  owner => principal_type name 
+    principal_type => STRING
+    name => STRING
+  issue_timestamp => INT64
+  expiry_timestamp => INT64
+  max_timestamp => INT64
+  token_id => STRING
+  hmac => BYTES
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
ownertoken owner.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
issue_timestamptimestamp (in msec) when this token was generated.
expiry_timestamptimestamp (in msec) at which this token expires.
max_timestampmax life time of this token.
token_idUUID to ensure uniqueness.
hmacHMAC of the delegation token.
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+
RenewDelegationToken API (Key: 39):
+ +Requests:
+

RenewDelegationToken Request (Version: 0) => hmac renew_time_period 
+  hmac => BYTES
+  renew_time_period => INT64
+
+ + + + + + +
FieldDescription
hmacHMAC of the delegation token to be renewed.
renew_time_periodRenew time period in milli seconds.
+

+

RenewDelegationToken Request (Version: 1) => hmac renew_time_period 
+  hmac => BYTES
+  renew_time_period => INT64
+
+ + + + + + +
FieldDescription
hmacHMAC of the delegation token to be renewed.
renew_time_periodRenew time period in milli seconds.
+

+Responses:
+

RenewDelegationToken Response (Version: 0) => error_code expiry_timestamp throttle_time_ms 
+  error_code => INT16
+  expiry_timestamp => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + +
FieldDescription
error_codeResponse error code
expiry_timestamptimestamp (in msec) at which this token expires..
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

RenewDelegationToken Response (Version: 1) => error_code expiry_timestamp throttle_time_ms 
+  error_code => INT16
+  expiry_timestamp => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + +
FieldDescription
error_codeResponse error code
expiry_timestamptimestamp (in msec) at which this token expires..
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+
ExpireDelegationToken API (Key: 40):
+ +Requests:
+

ExpireDelegationToken Request (Version: 0) => hmac expiry_time_period 
+  hmac => BYTES
+  expiry_time_period => INT64
+
+ + + + + + +
FieldDescription
hmacHMAC of the delegation token to be expired.
expiry_time_periodexpiry time period in milli seconds.
+

+

ExpireDelegationToken Request (Version: 1) => hmac expiry_time_period 
+  hmac => BYTES
+  expiry_time_period => INT64
+
+ + + + + + +
FieldDescription
hmacHMAC of the delegation token to be expired.
expiry_time_periodexpiry time period in milli seconds.
+

+Responses:
+

ExpireDelegationToken Response (Version: 0) => error_code expiry_timestamp throttle_time_ms 
+  error_code => INT16
+  expiry_timestamp => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + +
FieldDescription
error_codeResponse error code
expiry_timestamptimestamp (in msec) at which this token expires..
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

ExpireDelegationToken Response (Version: 1) => error_code expiry_timestamp throttle_time_ms 
+  error_code => INT16
+  expiry_timestamp => INT64
+  throttle_time_ms => INT32
+
+ + + + + + + + +
FieldDescription
error_codeResponse error code
expiry_timestamptimestamp (in msec) at which this token expires..
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+
DescribeDelegationToken API (Key: 41):
+ +Requests:
+

DescribeDelegationToken Request (Version: 0) => [owners] 
+  owners => principal_type name 
+    principal_type => STRING
+    name => STRING
+
+ + + + + + + + +
FieldDescription
ownersAn array of token owners.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
+

+

DescribeDelegationToken Request (Version: 1) => [owners] 
+  owners => principal_type name 
+    principal_type => STRING
+    name => STRING
+
+ + + + + + + + +
FieldDescription
ownersAn array of token owners.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
+

+Responses:
+

DescribeDelegationToken Response (Version: 0) => error_code [token_details] throttle_time_ms 
+  error_code => INT16
+  token_details => owner issue_timestamp expiry_timestamp max_timestamp token_id hmac [renewers] 
+    owner => principal_type name 
+      principal_type => STRING
+      name => STRING
+    issue_timestamp => INT64
+    expiry_timestamp => INT64
+    max_timestamp => INT64
+    token_id => STRING
+    hmac => BYTES
+    renewers => principal_type name 
+      principal_type => STRING
+      name => STRING
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
token_detailsnull
ownertoken owner.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
issue_timestamptimestamp (in msec) when this token was generated.
expiry_timestamptimestamp (in msec) at which this token expires.
max_timestampmax life time of this token.
token_idUUID to ensure uniqueness.
hmacHMAC of the delegation token to be expired.
renewersAn array of token renewers. Renewer is an Kafka PrincipalType and name string, who is allowed to renew this token before the max lifetime expires.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+

DescribeDelegationToken Response (Version: 1) => error_code [token_details] throttle_time_ms 
+  error_code => INT16
+  token_details => owner issue_timestamp expiry_timestamp max_timestamp token_id hmac [renewers] 
+    owner => principal_type name 
+      principal_type => STRING
+      name => STRING
+    issue_timestamp => INT64
+    expiry_timestamp => INT64
+    max_timestamp => INT64
+    token_id => STRING
+    hmac => BYTES
+    renewers => principal_type name 
+      principal_type => STRING
+      name => STRING
+  throttle_time_ms => INT32
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldDescription
error_codeResponse error code
token_detailsnull
ownertoken owner.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
issue_timestamptimestamp (in msec) when this token was generated.
expiry_timestamptimestamp (in msec) at which this token expires.
max_timestampmax life time of this token.
token_idUUID to ensure uniqueness.
hmacHMAC of the delegation token to be expired.
renewersAn array of token renewers. Renewer is an Kafka PrincipalType and name string, who is allowed to renew this token before the max lifetime expires.
principal_typeprincipalType of the Kafka principal
namename of the Kafka principal
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
+

+
DeleteGroups API (Key: 42):
+ +Requests:
+

DeleteGroups Request (Version: 0) => [groups] 
+  groups => STRING
+
+ + + + +
FieldDescription
groupsAn array of groups to be deleted.
+

+

DeleteGroups Request (Version: 1) => [groups] 
+  groups => STRING
+
+ + + + +
FieldDescription
groupsAn array of groups to be deleted.
+

+Responses:
+

DeleteGroups Response (Version: 0) => throttle_time_ms [group_error_codes] 
+  throttle_time_ms => INT32
+  group_error_codes => group_id error_code 
+    group_id => STRING
+    error_code => INT16
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
group_error_codesAn array of per group error codes.
group_idThe unique group identifier
error_codeResponse error code
+

+

DeleteGroups Response (Version: 1) => throttle_time_ms [group_error_codes] 
+  throttle_time_ms => INT32
+  group_error_codes => group_id error_code 
+    group_id => STRING
+    error_code => INT16
+
+ + + + + + + + + + +
FieldDescription
throttle_time_msDuration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)
group_error_codesAn array of per group error codes.
group_idThe unique group identifier
error_codeResponse error code
+

+ diff --git a/21/generated/protocol_types.html b/21/generated/protocol_types.html index e69de29..5b34cb9 100644 --- a/21/generated/protocol_types.html +++ b/21/generated/protocol_types.html @@ -0,0 +1,20 @@ + + + + + + + + + + + + + + + + + + +
TypeDescription
BOOLEANRepresents a boolean value in a byte. Values 0 and 1 are used to represent false and true respectively. When reading a boolean value, any non-zero value is considered true.
INT8Represents an integer between -27 and 27-1 inclusive.
INT16Represents an integer between -215 and 215-1 inclusive. The values are encoded using two bytes in network byte order (big-endian).
INT32Represents an integer between -231 and 231-1 inclusive. The values are encoded using four bytes in network byte order (big-endian).
INT64Represents an integer between -263 and 263-1 inclusive. The values are encoded using eight bytes in network byte order (big-endian).
UINT32Represents an integer between 0 and 232-1 inclusive. The values are encoded using four bytes in network byte order (big-endian).
VARINTRepresents an integer between -231 and 231-1 inclusive. Encoding follows the variable-length zig-zag encoding from Google Protocol Buffers.
VARLONGRepresents an integer between -263 and 263-1 inclusive. Encoding follows the variable-length zig-zag encoding from Google Protocol Buffers.
STRINGRepresents a sequence of characters. First the length N is given as an INT16. Then N bytes follow which are the UTF-8 encoding of the character sequence. Length must not be negative.
NULLABLE_STRINGRepresents a sequence of characters or null. For non-null strings, first the length N is given as an INT16. Then N bytes follow which are the UTF-8 encoding of the character sequence. A null value is encoded with length of -1 and there are no following bytes.
BYTESRepresents a raw sequence of bytes. First the length N is given as an INT32. Then N bytes follow.
NULLABLE_BYTESRepresents a raw sequence of bytes or null. For non-null values, first the length N is given as an INT32. Then N bytes follow. A null value is encoded with length of -1 and there are no following bytes.
RECORDSRepresents a sequence of Kafka records as NULLABLE_BYTES. For a detailed description of records see Message Sets.
ARRAYRepresents a sequence of objects of a given type T. Type T can be either a primitive type (e.g. STRING) or a structure. First, the length N is given as an INT32. Then N instances of type T follow. A null array is represented with a length of -1. In protocol documentation an array of T instances is referred to as [T].
+ diff --git a/21/generated/sink_connector_config.html b/21/generated/sink_connector_config.html index e69de29..b14636e 100644 --- a/21/generated/sink_connector_config.html +++ b/21/generated/sink_connector_config.html @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
nameGlobally unique name to use for this connector.stringnon-empty string without ISO control charactershigh
connector.className or alias of the class for this connector. Must be a subclass of org.apache.kafka.connect.connector.Connector. If the connector is org.apache.kafka.connect.file.FileStreamSinkConnector, you can either specify this full name, or use "FileStreamSink" or "FileStreamSinkConnector" to make the configuration a bit shorterstringhigh
tasks.maxMaximum number of tasks to use for this connector.int1[1,...]high
topicsList of topics to consume, separated by commaslist""high
topics.regexRegular expression giving topics to consume. Under the hood, the regex is compiled to a java.util.regex.Pattern. Only one of topics or topics.regex should be specified.string""valid regexhigh
key.converterConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the keys in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro.classnulllow
value.converterConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the values in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro.classnulllow
header.converterHeaderConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the header values in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro. By default, the SimpleHeaderConverter is used to serialize header values to strings and deseria [...] +
config.action.reloadThe action that Connect should take on the connector when changes in external configuration providers result in a change in the connector's configuration properties. A value of 'none' indicates that Connect will do nothing. A value of 'restart' indicates that Connect should restart/reload the connector with the updated configuration properties.The restart may actually be scheduled in the future if the external configuration provider indicates that a [...] +
transformsAliases for the transformations to be applied to records.list""non-null string, unique transformation aliaseslow
errors.retry.timeoutThe maximum duration in milliseconds that a failed operation will be reattempted. The default is 0, which means no retries will be attempted. Use -1 for infinite retries.long0medium
errors.retry.delay.max.msThe maximum duration in milliseconds between consecutive retry attempts. Jitter will be added to the delay once this limit is reached to prevent thundering herd issues.long60000medium
errors.toleranceBehavior for tolerating errors during connector operation. 'none' is the default value and signals that any error will result in an immediate connector task failure; 'all' changes the behavior to skip over problematic records.stringnone[none, all]medium
errors.log.enableIf true, write each error and the details of the failed operation and problematic record to the Connect application log. This is 'false' by default, so that only errors that are not tolerated are reported.booleanfalsemedium
errors.log.include.messagesWhether to the include in the log the Connect record that resulted in a failure. This is 'false' by default, which will prevent record keys, values, and headers from being written to log files, although some information such as topic and partition number will still be logged.booleanfalsemedium
errors.deadletterqueue.topic.nameThe name of the topic to be used as the dead letter queue (DLQ) for messages that result in an error when processed by this sink connector, or its transformations or converters. The topic name is blank by default, which means that no messages are to be recorded in the DLQ.string""medium
errors.deadletterqueue.topic.replication.factorReplication factor used to create the dead letter queue topic when it doesn't already exist.short3medium
errors.deadletterqueue.context.headers.enableIf true, add headers containing error context to the messages written to the dead letter queue. To avoid clashing with headers from the original record, all error context header keys, all error context header keys will start with __connect.errors.booleanfalsemedium
diff --git a/21/generated/source_connector_config.html b/21/generated/source_connector_config.html index e69de29..b29bfab 100644 --- a/21/generated/source_connector_config.html +++ b/21/generated/source_connector_config.html @@ -0,0 +1,36 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
nameGlobally unique name to use for this connector.stringnon-empty string without ISO control charactershigh
connector.className or alias of the class for this connector. Must be a subclass of org.apache.kafka.connect.connector.Connector. If the connector is org.apache.kafka.connect.file.FileStreamSinkConnector, you can either specify this full name, or use "FileStreamSink" or "FileStreamSinkConnector" to make the configuration a bit shorterstringhigh
tasks.maxMaximum number of tasks to use for this connector.int1[1,...]high
key.converterConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the keys in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro.classnulllow
value.converterConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the values in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro.classnulllow
header.converterHeaderConverter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the header values in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro. By default, the SimpleHeaderConverter is used to serialize header values to strings and deseria [...] +
config.action.reloadThe action that Connect should take on the connector when changes in external configuration providers result in a change in the connector's configuration properties. A value of 'none' indicates that Connect will do nothing. A value of 'restart' indicates that Connect should restart/reload the connector with the updated configuration properties.The restart may actually be scheduled in the future if the external configuration provider indicates that a [...] +
transformsAliases for the transformations to be applied to records.list""non-null string, unique transformation aliaseslow
errors.retry.timeoutThe maximum duration in milliseconds that a failed operation will be reattempted. The default is 0, which means no retries will be attempted. Use -1 for infinite retries.long0medium
errors.retry.delay.max.msThe maximum duration in milliseconds between consecutive retry attempts. Jitter will be added to the delay once this limit is reached to prevent thundering herd issues.long60000medium
errors.toleranceBehavior for tolerating errors during connector operation. 'none' is the default value and signals that any error will result in an immediate connector task failure; 'all' changes the behavior to skip over problematic records.stringnone[none, all]medium
errors.log.enableIf true, write each error and the details of the failed operation and problematic record to the Connect application log. This is 'false' by default, so that only errors that are not tolerated are reported.booleanfalsemedium
errors.log.include.messagesWhether to the include in the log the Connect record that resulted in a failure. This is 'false' by default, which will prevent record keys, values, and headers from being written to log files, although some information such as topic and partition number will still be logged.booleanfalsemedium
diff --git a/21/generated/streams_config.html b/21/generated/streams_config.html index e69de29..255de5c 100644 --- a/21/generated/streams_config.html +++ b/21/generated/streams_config.html @@ -0,0 +1,88 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesImportance
application.idAn identifier for the stream processing application. Must be unique within the Kafka cluster. It is used as 1) the default client-id prefix, 2) the group-id for membership management, 3) the changelog topic prefix.stringhigh
bootstrap.serversA list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form host1:port1,host2:port2,.... Since these servers are just used for the initial connection to discover the full cluster me [...] +
replication.factorThe replication factor for change log topics and repartition topics created by the stream processing application.int1high
state.dirDirectory location for state store.string/tmp/kafka-streamshigh
cache.max.bytes.bufferingMaximum number of memory bytes to be used for buffering across all threadslong10485760[0,...]medium
client.idAn ID prefix string used for the client IDs of internal consumer, producer and restore-consumer, with pattern '-StreamThread--'.string""medium
default.deserialization.exception.handlerException handling class that implements the org.apache.kafka.streams.errors.DeserializationExceptionHandler interface.classorg.apache.kafka.streams.errors.LogAndFailExceptionHandlermedium
default.key.serde Default serializer / deserializer class for key that implements the org.apache.kafka.common.serialization.Serde interface. Note when windowed serde class is used, one needs to set the inner serde class that implements the org.apache.kafka.common.serialization.Serde interface via 'default.windowed.key.serde.inner' or 'default.windowed.value.serde.inner' as wellclassorg.apache.kafka.common.serializati [...] +
default.production.exception.handlerException handling class that implements the org.apache.kafka.streams.errors.ProductionExceptionHandler interface.classorg.apache.kafka.streams.errors.DefaultProductionExceptionHandlermedium
default.timestamp.extractorDefault timestamp extractor class that implements the org.apache.kafka.streams.processor.TimestampExtractor interface.classorg.apache.kafka.streams.processor.FailOnInvalidTimestampmedium
default.value.serdeDefault serializer / deserializer class for value that implements the org.apache.kafka.common.serialization.Serde interface. Note when windowed serde class is used, one needs to set the inner serde class that implements the org.apache.kafka.common.serialization.Serde interface via 'default.windowed.key.serde.inner' or 'default.windowed.value.serde.inner' as wellclassorg.apache.kafka.common.serializ [...] +
max.task.idle.msMaximum amount of time a stream task will stay idle when not all of its partition buffers contain records, to avoid potential out-of-order record processing across multiple input streams.long0medium
num.standby.replicasThe number of standby replicas for each task.int0medium
num.stream.threadsThe number of threads to execute stream processing.int1medium
processing.guaranteeThe processing guarantee that should be used. Possible values are at_least_once (default) and exactly_once. Note that exactly-once processing requires a cluster of at least three brokers by default what is the recommended setting for production; for development you can change this, by adjusting broker setting transaction.state.log.replication.factor and transaction.state.log.min.isr.st [...] +
security.protocolProtocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.stringPLAINTEXTmedium
topology.optimizationA configuration telling Kafka Streams if it should optimize the topology, disabled by defaultstringnone[none, all]medium
application.serverA host:port pair pointing to an embedded user defined endpoint that can be used for discovering the locations of state stores within a single KafkaStreams applicationstring""low
buffered.records.per.partitionMaximum number of records to buffer per partition.int1000low
commit.interval.msThe frequency with which to save the position of the processor. (Note, if processing.guarantee is set to exactly_once, the default value is 100, otherwise the default value is 30000.long30000low
connections.max.idle.msClose idle connections after the number of milliseconds specified by this config.long540000low
metadata.max.age.msThe period of time in milliseconds after which we force a refresh of metadata even if we haven't seen any partition leadership changes to proactively discover any new brokers or partitions.long300000[0,...]low
metric.reportersA list of classes to use as metrics reporters. Implementing the org.apache.kafka.common.metrics.MetricsReporter interface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics.list""low
metrics.num.samplesThe number of samples maintained to compute metrics.int2[1,...]low
metrics.recording.levelThe highest recording level for metrics.stringINFO[INFO, DEBUG]low
metrics.sample.window.msThe window of time a metrics sample is computed over.long30000[0,...]low
partition.grouperPartition grouper class that implements the org.apache.kafka.streams.processor.PartitionGrouper interface.classorg.apache.kafka.streams.processor.DefaultPartitionGrouperlow
poll.msThe amount of time in milliseconds to block waiting for input.long100low
receive.buffer.bytesThe size of the TCP receive buffer (SO_RCVBUF) to use when reading data. If the value is -1, the OS default will be used.int32768[-1,...]low
reconnect.backoff.max.msThe maximum amount of time in milliseconds to wait when reconnecting to a broker that has repeatedly failed to connect. If provided, the backoff per host will increase exponentially for each consecutive connection failure, up to this maximum. After calculating the backoff increase, 20% random jitter is added to avoid connection storms.long1000[0,...]low
reconnect.backoff.msThe base amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop. This backoff applies to all connection attempts by the client to a broker.long50[0,...]low
request.timeout.msThe configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.int40000[0,...]low
retriesSetting a value greater than zero will cause the client to resend any request that fails with a potentially transient error.int0[0,...,2147483647]low
retry.backoff.msThe amount of time to wait before attempting to retry a failed request to a given topic partition. This avoids repeatedly sending requests in a tight loop under some failure scenarios.long100[0,...]low
rocksdb.config.setterA Rocks DB config setter class or class name that implements the org.apache.kafka.streams.state.RocksDBConfigSetter interfaceclassnulllow
send.buffer.bytesThe size of the TCP send buffer (SO_SNDBUF) to use when sending data. If the value is -1, the OS default will be used.int131072[-1,...]low
state.cleanup.delay.msThe amount of time in milliseconds to wait before deleting state when a partition has migrated. Only state directories that have not been modified for at least state.cleanup.delay.ms will be removedlong600000low
upgrade.fromAllows upgrading from versions 0.10.0/0.10.1/0.10.2/0.11.0/1.0/1.1 to version 1.2 (or newer) in a backward compatible way. When upgrading from 1.2 to a newer version it is not required to specify this config.Default is null. Accepted values are "0.10.0", "0.10.1", "0.10.2", "0.11.0", "1.0", "1.1" (for upgrading from the corresponding old version).stringnull[null, 0.10.0, 0.10.1, 0.10.2, 0.11.0, 1.0, 1.1] +windowstore.changelog.additional.retention.msAdded to a windows maintainMs to ensure data is not deleted from the log prematurely. Allows for clock drift. Default is 1 daylong86400000low
diff --git a/21/generated/topic_config.html b/21/generated/topic_config.html index e69de29..24a9505 100644 --- a/21/generated/topic_config.html +++ b/21/generated/topic_config.html @@ -0,0 +1,61 @@ + + + + + + + + + + + + + + + + + + + + + [...] + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionTypeDefaultValid ValuesServer Default PropertyImportance
cleanup.policyA string that is either "delete" or "compact". This string designates the retention policy to use on old log segments. The default policy ("delete") will discard old segments when their retention time or size limit has been reached. The "compact" setting will enable log compaction on the topic.listdelete[compact, delete]log.cleanup.policymedium
compression.typeSpecify the final compression type for a given topic. This configuration accepts the standard compression codecs ('gzip', 'snappy', 'lz4', 'zstd'). It additionally accepts 'uncompressed' which is equivalent to no compression; and 'producer' which means retain the original compression codec set by the producer.stringproducer[uncompressed, zstd, lz4, snappy, gzip, producer]compression.typedelete.retention.msThe amount of time to retain delete tombstone markers for log compacted topics. This setting also gives a bound on the time in which a consumer must complete a read if they begin from offset 0 to ensure that they get a valid snapshot of the final stage (otherwise delete tombstones may be collected before they complete their scan).long86400000[0,...]log.cleaner.delete.ret [...] +
file.delete.delay.msThe time to wait before deleting a file from the filesystemlong60000[0,...]log.segment.delete.delay.msmedium
flush.messagesThis setting allows specifying an interval at which we will force an fsync of data written to the log. For example if this was set to 1 we would fsync after every message; if it were 5 we would fsync after every five messages. In general we recommend you not set this and use replication for durability and allow the operating system's background flush capabilities as it is more efficient. This setting can be overridden on a per-topic basis (see [partitionId],[brokerId]:[partitionId],[brokerId]:...follower.replication.throttled.replica [...] +
index.interval.bytesThis setting controls how frequently Kafka adds an index entry to its offset index. The default setting ensures that we index a message roughly every 4096 bytes. More indexing allows reads to jump closer to the exact position in the log but makes the index larger. You probably don't need to change this.int4096[0,...]log.index.interval.bytesmedium
leader.replication.throttled.replicasA list of replicas for which log replication should be throttled on the leader side. The list should describe a set of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively the wildcard '*' can be used to throttle all replicas for this topic.list""[partitionId],[brokerId]:[partitionId],[brokerId]:...leader.replication.throttled.replicas
max.message.bytes

The largest record batch size allowed by Kafka. If this is increased and there are consumers older than 0.10.2, the consumers' fetch size must also be increased so that the they can fetch record batches this large.

In the latest message format version, records are always grouped into batches for efficiency. In previous message format versions, uncompressed records are not grouped into batches and this limit only applies to a single record in th [...] +

message.format.versionSpecify the message format version the broker will use to append messages to the logs. The value should be a valid ApiVersion. Some examples are: 0.8.2, 0.9.0.0, 0.10.0, check ApiVersion for more details. By setting a particular message format version, the user is certifying that all the existing messages on disk are smaller or equal than the specified version. Setting this value incorrectly will cause consumers with older versions to break as they [...] +
message.timestamp.difference.max.msThe maximum difference allowed between the timestamp when a broker receives a message and the timestamp specified in the message. If message.timestamp.type=CreateTime, a message will be rejected if the difference in timestamp exceeds this threshold. This configuration is ignored if message.timestamp.type=LogAppendTime.long9223372036854775807[0,...]log.message.timestamp.difference. [...] +
message.timestamp.typeDefine whether the timestamp in the message is message create time or log append time. The value should be either `CreateTime` or `LogAppendTime`stringCreateTime[CreateTime, LogAppendTime]log.message.timestamp.typemedium
min.cleanable.dirty.ratioThis configuration controls how frequently the log compactor will attempt to clean the log (assuming log compaction is enabled). By default we will avoid cleaning a log where more than 50% of the log has been compacted. This ratio bounds the maximum space wasted in the log by duplicates (at 50% at most 50% of the log could be duplicates). A higher ratio will mean fewer, more efficient cleanings but will mean more wasted [...] +
min.compaction.lag.msThe minimum time a message will remain uncompacted in the log. Only applicable for logs that are being compacted.long0[0,...]log.cleaner.min.compaction.lag.msmedium
min.insync.replicasWhen a producer sets acks to "all" (or "-1"), this configuration specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend).
When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical sce [...] +
preallocateTrue if we should preallocate the file on disk when creating a new log segment.booleanfalselog.preallocatemedium
retention.bytesThis configuration controls the maximum size a partition (which consists of log segments) can grow to before we will discard old log segments to free up space if we are using the "delete" retention policy. By default there is no size limit only a time limit. Since this limit is enforced at the partition level, multiply it by the number of partitions to compute the topic retention in bytes.long-1log.r [...] +
retention.msThis configuration controls the maximum time we will retain a log before we will discard old log segments to free up space if we are using the "delete" retention policy. This represents an SLA on how soon consumers must read their data. If set to -1, no time limit is applied.long604800000log.retention.msmedium
segment.bytesThis configuration controls the segment file size for the log. Retention and cleaning is always done a file at a time so a larger segment size means fewer files but less granular control over retention.int1073741824[14,...]log.segment.bytesmedium
segment.index.bytesThis configuration controls the size of the index that maps offsets to file positions. We preallocate this index file and shrink it only after log rolls. You generally should not need to change this setting.int10485760[0,...]log.index.size.max.bytesmedium
segment.jitter.msThe maximum random jitter subtracted from the scheduled segment roll time to avoid thundering herds of segment rollinglong0[0,...]log.roll.jitter.msmedium
segment.msThis configuration controls the period of time after which Kafka will force the log to roll even if the segment file isn't full to ensure that retention can delete or compact old data.long604800000[1,...]log.roll.msmedium
unclean.leader.election.enableIndicates whether to enable replicas not in the ISR set to be elected as leader as a last resort, even though doing so may result in data loss.booleanfalseunclean.leader.election.enablemedium
message.downconversion.enableThis configuration controls whether down-conversion of message formats is enabled to satisfy consume requests. When set to false, broker will not perform down-conversion for consumers expecting an older message format. The broker responds with UNSUPPORTED_VERSION error for consume requests from such older clients. This configurationdoes not apply to any message format conversion that might be required for replicatio [...] +