kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maxime Brugidou (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-634) ConsoleProducer compresses messages and ignores the --compress flag
Date Wed, 09 Jan 2013 12:20:13 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Maxime Brugidou resolved KAFKA-634.
-----------------------------------

       Resolution: Fixed
    Fix Version/s: 0.8
    
> ConsoleProducer compresses messages and ignores the --compress flag
> -------------------------------------------------------------------
>
>                 Key: KAFKA-634
>                 URL: https://issues.apache.org/jira/browse/KAFKA-634
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.7
>            Reporter: Anentropic
>              Labels: console, producer
>             Fix For: 0.8
>
>
> I am using the kafka-producer-shell.sh script without the --compress option
> however my messages seem to be gzipped
> the docs say compression is off by default:
> http://incubator.apache.org/kafka/configuration.html
> The only producer.properties file I can find is at:
> /home/ubuntu/kafka-0.7.2-incubating-src/config/producer.properties
> In there is:
> compression.codec=0
> My process looks like:
> root      1748  1746  0 Nov19 ?        00:02:37 java -Xmx512M -server -Dlog4j.configuration=file:/usr/local/bin/kafka/../config/log4j.properties
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
-cp :/usr/local/bin/kafka/../project/boot/scala-2.8.0/lib/scala-compiler.jar:/usr/local/bin/kafka/../project/boot/scala-2.8.0/lib/scala-library.jar:/usr/local/bin/kafka/../core/target/scala_2.8.0/kafka-0.7.2.jar:/usr/local/bin/kafka/../core/lib/*.jar:/usr/local/bin/kafka/../perf/target/scala_2.8.0/kafka-perf-0.7.2.jar:/usr/local/bin/kafka/../core/lib_managed/scala_2.8.0/compile/jopt-simple-3.2.jar:/usr/local/bin/kafka/../core/lib_managed/scala_2.8.0/compile/log4j-1.2.15.jar:/usr/local/bin/kafka/../core/lib_managed/scala_2.8.0/compile/snappy-java-1.0.4.1.jar:/usr/local/bin/kafka/../core/lib_managed/scala_2.8.0/compile/zkclient-0.1.jar:/usr/local/bin/kafka/../core/lib_managed/scala_2.8.0/compile/zookeeper-3.3.4.jar
kafka.producer.ConsoleProducer --topic logtail --zookeeper x.x.x.x:2181
> But the messages come out gobbledegook unless I use a client that understands compressed
messages, and in that client it identifies the bit as set to 1, gzip compression.
> Jun Rao junrao@gmail.com via incubator.apache.org 
> Nov 26 (1 day ago)
> to kafka-users 
> This seems to be a bug in ConsoleProducer. It also compresses messages and
> ignores the --compress flag. Could you file a jira?
> Thanks,
> Jun

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message