Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Wed, 30 Jan 2013 16:41:13 +0000 (UTC)
From: "Jay Kreps (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.12626087.1357327194128.216025.1359564073475@arcas>
In-Reply-To: <JIRA.12626087.1357327194128@arcas>
References: <JIRA.12626087.1357327194128@arcas>
Subject: [jira] [Commented] (KAFKA-683) Fix correlation ids in all requests
 sent to kafka
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/KAFKA-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566599#comment-13566599 ] 

Jay Kreps commented on KAFKA-683:
---------------------------------

The design was that the network layer is fully generic and the interface it presents to the application code is basically a "queue" or "channel" to dequeue and enqueue requests and responses. All the network layer knows about requests and responses is that they are glorified byte arrays.

The details of how we handle request processing or the format of requests is not part of the network system. There is a specific layer, KafkaRequestHandler which is the  kafka part. Note how this bit happens in the *processing* thread pool not in the *network* thread pool.

The thought I had was to eventually factor more out of the KafkaApis layer and into the KafkaRequestHandler layer to make KafkaApis easier to test in the absence of a network server (i.e. you should just call new KafkaApis and then test the individual handle() methods). That was the layer that is meant to adopt our "business logic" in kafka apis to details of serialization, network, etc. Neha and I had briefly discussed this a few times.

I am not saying this plan is the right way to go, I just would have liked to discuss that approach. 
                
> Fix correlation ids in all requests sent to kafka
> -------------------------------------------------
>
>                 Key: KAFKA-683
>                 URL: https://issues.apache.org/jira/browse/KAFKA-683
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 0.8
>            Reporter: Neha Narkhede
>            Assignee: Neha Narkhede
>            Priority: Critical
>              Labels: improvement, replication
>         Attachments: kafka-683-v1.patch, kafka-683-v2.patch, kafka-683-v2-rebased.patch, kafka-683-v2-rebased-twice.patch, kafka-683-v3.patch, kafka-683-v3-unit-test.patch
>
>
> We should fix the correlation ids in every request sent to Kafka and fix the request log on the broker to specify not only the type of request and who sent it, but also the correlation id. This will be very helpful while troubleshooting problems in production.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira