kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay Kreps (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-683) Fix correlation ids in all requests sent to kafka
Date Wed, 30 Jan 2013 16:41:13 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566599#comment-13566599
] 

Jay Kreps commented on KAFKA-683:
---------------------------------

The design was that the network layer is fully generic and the interface it presents to the
application code is basically a "queue" or "channel" to dequeue and enqueue requests and responses.
All the network layer knows about requests and responses is that they are glorified byte arrays.

The details of how we handle request processing or the format of requests is not part of the
network system. There is a specific layer, KafkaRequestHandler which is the  kafka part. Note
how this bit happens in the *processing* thread pool not in the *network* thread pool.

The thought I had was to eventually factor more out of the KafkaApis layer and into the KafkaRequestHandler
layer to make KafkaApis easier to test in the absence of a network server (i.e. you should
just call new KafkaApis and then test the individual handle() methods). That was the layer
that is meant to adopt our "business logic" in kafka apis to details of serialization, network,
etc. Neha and I had briefly discussed this a few times.

I am not saying this plan is the right way to go, I just would have liked to discuss that
approach. 
                
> Fix correlation ids in all requests sent to kafka
> -------------------------------------------------
>
>                 Key: KAFKA-683
>                 URL: https://issues.apache.org/jira/browse/KAFKA-683
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 0.8
>            Reporter: Neha Narkhede
>            Assignee: Neha Narkhede
>            Priority: Critical
>              Labels: improvement, replication
>         Attachments: kafka-683-v1.patch, kafka-683-v2.patch, kafka-683-v2-rebased.patch,
kafka-683-v2-rebased-twice.patch, kafka-683-v3.patch, kafka-683-v3-unit-test.patch
>
>
> We should fix the correlation ids in every request sent to Kafka and fix the request
log on the broker to specify not only the type of request and who sent it, but also the correlation
id. This will be very helpful while troubleshooting problems in production.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message