Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Wed, 30 Jan 2013 07:03:14 +0000 (UTC)
From: "Jay Kreps (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.12626087.1357327194128.214025.1359529394346@arcas>
In-Reply-To: <JIRA.12626087.1357327194128@arcas>
References: <JIRA.12626087.1357327194128@arcas>
Subject: [jira] [Commented] (KAFKA-683) Fix correlation ids in all requests
 sent to kafka
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/KAFKA-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566234#comment-13566234 ] 

Jay Kreps commented on KAFKA-683:
---------------------------------

Yeah, I am on the fence about the metrics stuff, maybe it does make sense to measure request metrics based on entrance and exit from the queue though clearly that implementation is super super nasty (I think we could make request metrics a general concept of name/start/stop and track them up and down the stack without too much unpleasentness).

But I still see on 0.8 the thing I am complaining about which is
  val requestObj: RequestOrResponse = RequestKeys.deserializerForKey(requestId)(buffer)
which if I understand correctly moves api deserialization into the network layer and into the network threads. This was not meant to be the definition of the network layer. It hard-codes our api set and format so you can no longer test that layer independently.

But basically yes, I was doing the code review for the other bug and noticed that indeed a virtually everyone on the team has come and done something mean to my beautiful little request queue (only Swapnil is innocent!). Each of these things was done as part of a useful feature but since it was done by someone who doesn't care much about this layer it was done without thinking about whether a request channel is a thing that instantiates producer requests, contains the list of all the metrics we track, parses requests, or whatever. I think each person was just trying to get their thing done and thought, ah, here is a nice place to shove my stuff. Probably everyone after the first person thought, "this code is real crap so no sense trying to hard" because once you have one or two WTFs in there no one else really puts in much effort.
                
> Fix correlation ids in all requests sent to kafka
> -------------------------------------------------
>
>                 Key: KAFKA-683
>                 URL: https://issues.apache.org/jira/browse/KAFKA-683
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 0.8
>            Reporter: Neha Narkhede
>            Assignee: Neha Narkhede
>            Priority: Critical
>              Labels: improvement, replication
>         Attachments: kafka-683-v1.patch, kafka-683-v2.patch, kafka-683-v2-rebased.patch, kafka-683-v2-rebased-twice.patch, kafka-683-v3.patch, kafka-683-v3-unit-test.patch
>
>
> We should fix the correlation ids in every request sent to Kafka and fix the request log on the broker to specify not only the type of request and who sent it, but also the correlation id. This will be very helpful while troubleshooting problems in production.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira