hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy Battula <brahmareddy.batt...@huawei.com>
Subject RE: FileSystem Vs ZKStateStore for RM recovery
Date Fri, 13 Feb 2015 09:07:10 GMT
Yes, you can configure yarn.resourcemanager.max-completed-applications based on your usage..All
the applications( RMStateStore ) will be stored in the ZK.As Tsuyoshi mentioned Zk will support..




Thanks & Regards
Brahma Reddy Battula
________________________________________
From: Tsuyoshi Ozawa [ozawa@apache.org]
Sent: Friday, February 13, 2015 12:30 PM
To: Tsuyoshi Ozawa
Cc: user@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: FileSystem Vs ZKStateStore for RM recovery

> I think ZooKeeper can handle thousands of updates,

I meant "thousands of updates per second".

Thanks,
- Tsuyoshi

On Fri, Feb 13, 2015 at 3:59 PM, Tsuyoshi Ozawa <ozawa@apache.org> wrote:
> Hi Suma,
>
> I think ZooKeeper can handle thousands of updates, so thousands of
> jobs can be "launched" at the same time.
> More jobs can be running at the same time since the number of updates
> against ZooKeeper is less than the number of jobs. Please free to ask
> us if you face the scalability or performance issue when you test. We
> can tackle the issue.
>
> Thanks,
> - Tsuyoshi
>
> On Wed, Feb 11, 2015 at 6:08 PM, Suma Shivaprasad
> <sumasai.shivaprasad@gmail.com> wrote:
>> Can ZKStateStore scale for large clusters. Any idea on the number of
>> concurrent jobs that can be supported on top of these ?
>>
>> Thanks
>> Suma
>>
>> On Wed, Feb 11, 2015 at 1:45 PM, Karthik Kambatla <kasha@cloudera.com>
>> wrote:
>>>
>>> We recommend ZK-store, particularly if you plan to deploy multiple
>>> ResourceManagers with failover. ZK-store ensures a single RM has write
>>> access and thus is better protected against split-brain cases where both RMs
>>> think they are active.
>>>
>>> On Tue, Feb 10, 2015 at 9:59 PM, Suma Shivaprasad
>>> <sumasai.shivaprasad@gmail.com> wrote:
>>>>
>>>> We are planning to deploy Hadoop 2.6.0 with a default configuration to
>>>> cache 10000 entries in the state store. With a workload of 150-250
>>>> concurrent applications at any time , which state store is better to use
>>>> and for what reasons ?
>>>>
>>>> Thanks
>>>> Suma
>>>
>>>
>>>
>>>
>>> --
>>> Karthik Kambatla
>>> Software Engineer, Cloudera Inc.
>>> --------------------------------------------
>>> http://five.sentenc.es
>>>
>>

Mime
View raw message