kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay Kreps (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-683) Fix correlation ids in all requests sent to kafka
Date Wed, 30 Jan 2013 07:03:14 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566234#comment-13566234
] 

Jay Kreps commented on KAFKA-683:
---------------------------------

Yeah, I am on the fence about the metrics stuff, maybe it does make sense to measure request
metrics based on entrance and exit from the queue though clearly that implementation is super
super nasty (I think we could make request metrics a general concept of name/start/stop and
track them up and down the stack without too much unpleasentness).

But I still see on 0.8 the thing I am complaining about which is
  val requestObj: RequestOrResponse = RequestKeys.deserializerForKey(requestId)(buffer)
which if I understand correctly moves api deserialization into the network layer and into
the network threads. This was not meant to be the definition of the network layer. It hard-codes
our api set and format so you can no longer test that layer independently.

But basically yes, I was doing the code review for the other bug and noticed that indeed a
virtually everyone on the team has come and done something mean to my beautiful little request
queue (only Swapnil is innocent!). Each of these things was done as part of a useful feature
but since it was done by someone who doesn't care much about this layer it was done without
thinking about whether a request channel is a thing that instantiates producer requests, contains
the list of all the metrics we track, parses requests, or whatever. I think each person was
just trying to get their thing done and thought, ah, here is a nice place to shove my stuff.
Probably everyone after the first person thought, "this code is real crap so no sense trying
to hard" because once you have one or two WTFs in there no one else really puts in much effort.
                
> Fix correlation ids in all requests sent to kafka
> -------------------------------------------------
>
>                 Key: KAFKA-683
>                 URL: https://issues.apache.org/jira/browse/KAFKA-683
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 0.8
>            Reporter: Neha Narkhede
>            Assignee: Neha Narkhede
>            Priority: Critical
>              Labels: improvement, replication
>         Attachments: kafka-683-v1.patch, kafka-683-v2.patch, kafka-683-v2-rebased.patch,
kafka-683-v2-rebased-twice.patch, kafka-683-v3.patch, kafka-683-v3-unit-test.patch
>
>
> We should fix the correlation ids in every request sent to Kafka and fix the request
log on the broker to specify not only the type of request and who sent it, but also the correlation
id. This will be very helpful while troubleshooting problems in production.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message