hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Saxena <vsaxena.va...@gmail.com>
Subject Re: Yarn ResourceManager web UI does not show job
Date Mon, 21 Sep 2015 17:23:32 GMT
No, you cant show them in RM UI then.

However if you can start another daemon, you can consider using YARN
Application History/Timeline Server or MR Job History Server(only for MR
jobs)  to see information about completed jobs.
You can look up Hadoop documentation to learn more about them and how to
configure them.

Just to clarify though, the apps themselves are not lost, as in, the output
is not lost. Its just the information about them which is no longer present
on RM restart.

Varun Saxena.

On Mon, Sep 21, 2015 at 10:31 PM, Boyu Zhang <boyuzhang35@gmail.com> wrote:

> Thanks for the answer Varun.
> It is the case that yarn.resourcemanager.recovery.enabled is set to be
> false. Is there a way to show the jobs that are submitted before the
> restart? We don't want to lose that data.
> Thanks,
> Boyu
> On Mon, Sep 21, 2015 at 12:53 PM, Varun Saxena <vsaxena.varun@gmail.com>
> wrote:
>> Hi Boyu,
>> RM stores apps in state store if recovery is enabled. Only then they will
>> be available on restart.
>> Otherwise they are kept in memory and hence lost on restart.
>> You may not have it enabled. Check config value for below config. By
>> default its false.
>> yarn.resourcemanager.recovery.enabled
>> Regards,
>> Varun.
>> On Mon, Sep 21, 2015 at 10:01 PM, Boyu Zhang <boyuzhang35@gmail.com>
>> wrote:
>>> Hello Everyone,
>>> I have a strange error regarding the ResourceManager web UI (
>>> http://xx.xx:8088).
>>> Someone before me set up the hadoop + yarn cluster using Pivotal HD, it
>>> was running fine. Then today, the resource manager and node manager
>>> disappeared, the logs did not record this. I restarted them, they are up
>>> and running, but the resource manger web UI does not show any jobs. We have
>>> 700+ jobs in the past, and they were showing before.
>>> If I submit MapReduce jobs, the new submitted ones show up. But the
>>> disappear again after restart the resource manger and node manager.
>>> Can anyone give any hint on where to look?
>>> Thanks in advance,
>>> Boyu

View raw message