hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krishna Kishore Bonagiri <write2kish...@gmail.com>
Subject Re: Time taken for starting AMRMClientAsync
Date Tue, 22 Oct 2013 09:39:18 GMT
Hi Alejandro,

  I submit all my applications from a single Client, but all of my
application masters are taking almost the same amount of time for finishing
the above calls. Do you reuse ApplicationMaster instances or do some other
thing for saving this time? Otherwise I felt the fresh application
connecting to the resource manager would take the same amount of time
although I don't know why should it take that much?

Thanks,
Kishore


On Mon, Oct 21, 2013 at 9:23 PM, Alejandro Abdelnur <tucu@cloudera.com>wrote:

> Hi Krishna,
>
> Those 900ms seems consistent with the numbers we  found while doing some
> benchmarks in the context of Llama:
>
> http://cloudera.github.io/llama/
>
> "We found that the first application master created from a client process
> takes around 900 ms to be ready to submit resource requests. Subsequent
> application masters created from the same client process take a mean of 20
> ms. The application master submission throughput (discarding the first
> submission) tops at approximately 100 application masters per second."
>
> I believe there is room for improvement there.
>
> Cheers
>
>
> On Mon, Oct 21, 2013 at 7:16 AM, Krishna Kishore Bonagiri <
> write2kishore@gmail.com> wrote:
>
>> Hi,
>>   I am seeing the following call to start() on AMRMClientAsync taking
>> from 0.9 to 1 second. Why does it take that long? Is there a way to reduce
>> it, I mean does it depend on any of the interval parameters or so in
>> configuration files? I have tried reducing the value of the first argument
>> below from 1000 to 100 seconds also, but that doesn't help.
>>
>>     AMRMClientAsync.CallbackHandler allocListener = new
>> RMCallbackHandler();
>>     amRMClient = AMRMClientAsync.createAMRMClientAsync(1000,
>> allocListener);
>>     amRMClient.init(conf);
>>     amRMClient.start();
>>
>>
>> Thanks,
>> Kishore
>>
>>
>
>
> --
> Alejandro
>

Mime
View raw message