kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haruki Okada <ocadar...@gmail.com>
Subject Re: Can't find any sendfile system call trace from Kafka process?
Date Thu, 03 Sep 2020 01:17:07 GMT
There are two cases that zero-copy fetch thanks to sendfile don't work.

- SSL encryption is enabled
  * Need to encrypt on Kafka process before sending to client
      -
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/network/SslTransportLayer.java#L946
  * Unlike plaintext transport layer which directly write to socket:
      -
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/network/PlaintextTransportLayer.java#L214
- Message down conversion happens
  * refs:
https://kafka.apache.org/documentation/#upgrade_10_performance_impact

Does your environment match above cases?

2020年9月1日(火) 9:05 Ming Liu <mingaliu@gmail.com>:

> Hi Kafka dev community,
>      As we know, one major reason that Kafka is fast is because it is using
> sendfile() for zero copy, as what it described at
> https://kafka.apache.org/documentation/#producerconfigs,
>
>
>
> *This combination of pagecache and sendfile means that on a Kafka cluster
> where the consumers are mostly caught up you will see no read activity on
> the disks whatsoever as they will be serving data entirely from cache.*
>
>     However, when I ran tracing on all my kafka brokers, I didn't get a
> single sendfile system call, why is this? Does it eventually translate to
> plain read/write syscalls?
>
> sudo ./syscount -p 126806 -d 30
> Tracing syscalls, printing top 10... Ctrl+C to quit.
> [17:44:10]
> SYSCALL          COUNT
> epoll_wait        108482
> write          107165
> epoll_ctl         95058
> futex           86716
> read           86388
> pread           26910
> fstat           9213
> getrusage          120
> close            27
> open             21
>


-- 
========================
Okada Haruki
ocadaruma@gmail.com
========================

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message