cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Roth <benjamin.r...@jaumo.com>
Subject Re: High disk io read load
Date Sat, 18 Feb 2017 20:49:50 GMT
256 tokens:

root@cas9:/sys/block/dm-0# blockdev --report
RO    RA   SSZ   BSZ   StartSec            Size   Device
rw   256   512  4096          0        67108864   /dev/ram0
rw   256   512  4096          0        67108864   /dev/ram1
rw   256   512  4096          0        67108864   /dev/ram2
rw   256   512  4096          0        67108864   /dev/ram3
rw   256   512  4096          0        67108864   /dev/ram4
rw   256   512  4096          0        67108864   /dev/ram5
rw   256   512  4096          0        67108864   /dev/ram6
rw   256   512  4096          0        67108864   /dev/ram7
rw   256   512  4096          0        67108864   /dev/ram8
rw   256   512  4096          0        67108864   /dev/ram9
rw   256   512  4096          0        67108864   /dev/ram10
rw   256   512  4096          0        67108864   /dev/ram11
rw   256   512  4096          0        67108864   /dev/ram12
rw   256   512  4096          0        67108864   /dev/ram13
rw   256   512  4096          0        67108864   /dev/ram14
rw   256   512  4096          0        67108864   /dev/ram15
rw    16   512  4096          0    800166076416   /dev/sda
rw    16   512  4096       2048    800164151296   /dev/sda1
rw    16   512  4096          0    644245094400   /dev/dm-0
rw    16   512  4096          0      2046820352   /dev/dm-1
rw    16   512  4096          0      1023410176   /dev/dm-2
rw    16   512  4096          0    800166076416   /dev/sdb

512 tokens:
root@cas10:/sys/block# blockdev --report
RO    RA   SSZ   BSZ   StartSec            Size   Device
rw   256   512  4096          0        67108864   /dev/ram0
rw   256   512  4096          0        67108864   /dev/ram1
rw   256   512  4096          0        67108864   /dev/ram2
rw   256   512  4096          0        67108864   /dev/ram3
rw   256   512  4096          0        67108864   /dev/ram4
rw   256   512  4096          0        67108864   /dev/ram5
rw   256   512  4096          0        67108864   /dev/ram6
rw   256   512  4096          0        67108864   /dev/ram7
rw   256   512  4096          0        67108864   /dev/ram8
rw   256   512  4096          0        67108864   /dev/ram9
rw   256   512  4096          0        67108864   /dev/ram10
rw   256   512  4096          0        67108864   /dev/ram11
rw   256   512  4096          0        67108864   /dev/ram12
rw   256   512  4096          0        67108864   /dev/ram13
rw   256   512  4096          0        67108864   /dev/ram14
rw   256   512  4096          0        67108864   /dev/ram15
rw    16   512  4096          0    800166076416   /dev/sda
rw    16   512  4096       2048    800164151296   /dev/sda1
rw    16   512  4096          0    800166076416   /dev/sdb
rw    16   512  4096       2048    800165027840   /dev/sdb1
rw    16   512  4096          0   1073741824000   /dev/dm-0
rw    16   512  4096          0      2046820352   /dev/dm-1
rw    16   512  4096          0      1023410176   /dev/dm-2

2017-02-18 21:41 GMT+01:00 Bhuvan Rawal <bhu1rawal@gmail.com>:

> Hi Ben,
>
> If its same on both machines then something else could be the issue. We
> faced high disk io due to misconfigured read ahead which resulted in high
> amount of disk io for comparatively insignificant network transfer.
>
> Can you post output of blockdev --report for a normal node and 512 token
> node.
>
> Regards,
>
> On Sun, Feb 19, 2017 at 2:07 AM, Benjamin Roth <benjamin.roth@jaumo.com>
> wrote:
>
>> cat /sys/block/sda/queue/read_ahead_kb
>> => 8
>>
>> On all CS nodes. Is that what you mean?
>>
>> 2017-02-18 21:32 GMT+01:00 Bhuvan Rawal <bhu1rawal@gmail.com>:
>>
>>> Hi Benjamin,
>>>
>>> What is the disk read ahead on both nodes?
>>>
>>> Regards,
>>> Bhuvan
>>>
>>> On Sun, Feb 19, 2017 at 1:58 AM, Benjamin Roth <benjamin.roth@jaumo.com>
>>> wrote:
>>>
>>>> This is status of the largest KS of these both nodes:
>>>> UN  10.23.71.10  437.91 GiB  512          49.1%
>>>> 2679c3fa-347e-4845-bfc1-c4d0bc906576  RAC1
>>>> UN  10.23.71.9   246.99 GiB  256          28.3%
>>>> 2804ef8a-26c8-4d21-9e12-01e8b6644c2f  RAC1
>>>>
>>>> So roughly as expected.
>>>>
>>>> 2017-02-17 23:07 GMT+01:00 kurt greaves <kurt@instaclustr.com>:
>>>>
>>>>> what's the Owns % for the relevant keyspace from nodetool status?
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Benjamin Roth
>>>> Prokurist
>>>>
>>>> Jaumo GmbH · www.jaumo.com
>>>> Wehrstraße 46 · 73035 Göppingen · Germany
>>>> Phone +49 7161 304880-6 <07161%203048806> · Fax +49 7161 304880-1
>>>> <07161%203048801>
>>>> AG Ulm · HRB 731058 · Managing Director: Jens Kammerer
>>>>
>>>
>>>
>>
>>
>> --
>> Benjamin Roth
>> Prokurist
>>
>> Jaumo GmbH · www.jaumo.com
>> Wehrstraße 46 · 73035 Göppingen · Germany
>> Phone +49 7161 304880-6 <07161%203048806> · Fax +49 7161 304880-1
>> <07161%203048801>
>> AG Ulm · HRB 731058 · Managing Director: Jens Kammerer
>>
>
>


-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Mime
View raw message