kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michal Borowiecki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4971) Why is there no difference between kafka benchmark tests on SSD and HDD?
Date Tue, 04 Apr 2017 15:34:42 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15955260#comment-15955260

Michal Borowiecki commented on KAFKA-4971:

I'd venture a guess that you are limited by something else than your hdd/ssd performance.
Is 1g your total memory in the VM? How much of it is allocated to the kafka jvm process?
Some things I can think of:
Is there a lot of activity in the gc.log?
Is the OS not swapping ferociously due to over-allocation of memory by any chance?

Hope that helps.

> Why is there no difference between kafka benchmark tests on SSD and HDD? 
> -------------------------------------------------------------------------
>                 Key: KAFKA-4971
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4971
>             Project: Kafka
>          Issue Type: Test
>    Affects Versions:
>         Environment: Oracle VM VirtualBox
> OS : CentOs 7
> Memory : 1G
> Disk : 8GB
>            Reporter: Dasol Kim
> I installed OS and kafka in the two SSD and two HDDs  to perform the kafka benchmark
test based on the disc difference. As expected, the SSD should show faster results, but according
to my experimental results, there is no big difference between SSD and HDD. why? Ohter settings
have been set to default.
> *test settings
> zookeeper node  : 1, producer node : 2, broker node : 2(SSD 1, HDD 1)
> test scenario : Two producers send messages to the broker and compare the throughtput
per second of kafka installed on SSD and kafka on HDD
> command : ./bin/kafka-producer-perf-test.sh --num-records 1000000 --record-size 2000
--topic test --throughput 100000 --producer-props bootstrap.servers=SN02:9092

This message was sent by Atlassian JIRA

View raw message