kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Koshy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-391) Producer request and response classes should use maps
Date Wed, 12 Sep 2012 01:35:07 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Joel Koshy updated KAFKA-391:

    Attachment: KAFKA-391-v2.patch

2.1 - Rebased after the first review, but I will need to rebase again. Unit
  tests and system tests pass. This patch applies cleanly to svn revision
  1381858 - it would be great if this can be reviewed against that revision.   I can provide
an incremental patch if that helps after I rebase.

2.2 - Removed all the equals methods in classes that were using Arrays earlier.

2.3 - Switched fetch request/response to maps. It makes the code a little
  cleaner, but I think the improvement was greater with the refactoring on
  the producer side. Anyway, this makes the APIs more consistent.  One small
  observation: previously, fetch requests would throw a
  FetchRequestFormatException if the TopicData array for a fetch request
  contained the same topic multiple times. Right now we don't do any checks
  since we use a map. We can add it to the FetchRequestBuilder, but not sure
  if it is required/worth it. Also, I think it previously would allow
  fetches from different offsets in the same partition in the same fetch
  request. That is no longer allowed, although I don't know why anyone would
  need that.

2.4 - Removed the global error code.

Another minor detail: I wonder if it would help to have a case class for
TopicAndPartition. We use (topic, partition) and the associated tuple
addressing all over the place. That would make the Map declarations slightly
clearer, although now we're accustomed to understanding that (String, Int)
must mean (topic, partitionId).

> Producer request and response classes should use maps
> -----------------------------------------------------
>                 Key: KAFKA-391
>                 URL: https://issues.apache.org/jira/browse/KAFKA-391
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Joel Koshy
>            Assignee: Joel Koshy
>              Labels: optimization
>             Fix For: 0.8
>         Attachments: KAFKA-391-draft-r1374069.patch, KAFKA-391-v2.patch
> Producer response contains two arrays of error codes and offsets - the ordering in these
arrays correspond to the flattened ordering of the request arrays.
> It would be better to switch to maps in the request and response as this would make the
code clearer and more efficient (right now, linear scans are used in handling producer acks).
> We can probably do the same in the fetch request/response.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message