hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 杨浩 <yangha...@gmail.com>
Subject Re: How to find bottlenecks of the cluster ?
Date Tue, 03 Mar 2015 13:19:31 GMT
I think benchmark will do some help, since it can help to find out the
executing speed of I/O rated job and CPU rated job

2015-03-02 19:01 GMT+08:00 Adrien Mogenet <adrien.mogenet@contentsquare.com>
:

> This is a non-sense ; you have to tell us under which conditions you want
> to find a bottleneck.
>
> Regardless the workload, we mostly use OpenTSDB to check cpu times (iowait
> / user / sys / idle), disk usage (await, ios in progress...) and memory
> (numa allocations, buffers, cache, dirty pages...)
>
> On 2 March 2015 at 08:20, Krish Donald <gotomypc27@gmail.com> wrote:
>
>> Basically we have 4 points to consider, CPU , Memory, IO and Network
>>
>> So how to see which one is causing the bottleneck ?
>> What parameters we should consider etc ?
>>
>> On Sun, Mar 1, 2015 at 10:57 PM, Nishanth S <nishanth.2884@gmail.com>
>> wrote:
>>
>>> This is a  vast  topic.Can you tell what components are there in your
>>> data pipe line and how data flows in to system and the way its
>>> processed.There are several  inbuilt tests like testDFSIO and terasort that
>>> you can run.
>>>
>>> -Nishan
>>>
>>> On Sun, Mar 1, 2015 at 9:45 PM, Krish Donald <gotomypc27@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I wanted to understand, how should we find out the bottleneck of the
>>>> cluster?
>>>>
>>>> Thanks
>>>> Krish
>>>>
>>>
>>>
>>
>
>
> --
>
> *Adrien Mogenet*
> Head of Backend/Infrastructure
> adrien.mogenet@contentsquare.com
> (+33)6.59.16.64.22
> http://www.contentsquare.com
> 4, avenue Franklin D. Roosevelt - 75008 Paris
>

Mime
View raw message