kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neha Narkhede (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-171) Kafka producer should do a single write to send message sets
Date Tue, 01 Nov 2011 05:10:32 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140935#comment-13140935
] 

Neha Narkhede commented on KAFKA-171:
-------------------------------------

nnarkhed-mn:kafka-171 nnarkhed$ find . -name "*scala" -exec grep -Hi "asInstanceOf\[Int\]"
{} \;
./core/src/main/scala/kafka/api/OffsetRequest.scala:  header.putInt(size.asInstanceOf[Int]
+ 2)
./core/src/main/scala/kafka/api/ProducerRequest.scala:  def sizeInBytes(): Int = 2 + topic.length
+ 4 + 4 + messages.sizeInBytes.asInstanceOf[Int]
./core/src/main/scala/kafka/network/BoundedByteBufferSend.scala:    written.asInstanceOf[Int]
./core/src/main/scala/kafka/producer/SyncProducer.scala:    val setSize = messages.sizeInBytes.asInstanceOf[Int]
./core/src/main/scala/kafka/server/MessageSetSend.scala:  header.putInt(size.asInstanceOf[Int]
+ 2)
./core/src/main/scala/kafka/server/MessageSetSend.scala:      written += fileBytesSent.asInstanceOf[Int]
./core/src/main/scala/kafka/server/MessageSetSend.scala:  def sendSize: Int = size.asInstanceOf[Int]
+ header.capacity
./core/src/main/scala/kafka/utils/Utils.scala:    buffer.putInt((value & 0xffffffffL).asInstanceOf[Int])
./core/src/main/scala/kafka/utils/Utils.scala:    buffer.putInt(index, (value & 0xffffffffL).asInstanceOf[Int])

Its not great that we have so many places where we need to worry about coercion, but we can
clean this up the next time we change the on wire protocol.

+1 on the latest patch
                
> Kafka producer should do a single write to send message sets
> ------------------------------------------------------------
>
>                 Key: KAFKA-171
>                 URL: https://issues.apache.org/jira/browse/KAFKA-171
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.7, 0.8
>            Reporter: Jay Kreps
>            Assignee: Jay Kreps
>             Fix For: 0.8
>
>         Attachments: KAFKA-171-draft.patch, KAFKA-171-v2.patch, KAFKA-171.patch
>
>
> From email thread: 
> http://mail-archives.apache.org/mod_mbox/incubator-kafka-dev/201110.mbox/%3cCAFbh0Q1PYUj32thBaYQ29E6J4wT_mrG5SuUsfdeGWj6rmEx9Gw@mail.gmail.com%3e
> > Before sending an actual message, kafka producer do send a (control) message of
4 bytes to the server. Kafka producer always does this action before send some message to
the server.
> I think this is because in BoundedByteBufferSend.scala we do essentially
>  channel.write(sizeBuffer)
>  channel.write(dataBuffer)
> The correct solution is to use vector I/O and instead do
>  channel.write(Array(sizeBuffer, dataBuffer))

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message