Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9DCC418D9F for ; Mon, 21 Sep 2015 17:23:45 +0000 (UTC) Received: (qmail 17037 invoked by uid 500); 21 Sep 2015 17:23:41 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 16928 invoked by uid 500); 21 Sep 2015 17:23:41 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 16918 invoked by uid 99); 21 Sep 2015 17:23:40 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Sep 2015 17:23:40 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 4CDFBC08AC for ; Mon, 21 Sep 2015 17:23:40 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.18 X-Spam-Level: ** X-Spam-Status: No, score=2.18 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id R3lgE2pqswej for ; Mon, 21 Sep 2015 17:23:39 +0000 (UTC) Received: from mail-ob0-f179.google.com (mail-ob0-f179.google.com [209.85.214.179]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id E323042B42 for ; Mon, 21 Sep 2015 17:23:38 +0000 (UTC) Received: by obbzf10 with SMTP id zf10so87305604obb.2 for ; Mon, 21 Sep 2015 10:23:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=/lclP3NkQVoKRnahzhy59SiDkhyQEap0ULT/WShcoZc=; b=BAehdwtD2fheVN/z2LA52AW47HN+bHKNup5w38inpr1oK1jI6nMrjaC14qEAK0pZCf QfYaFB3FQKDRdVEDiaXfxJQo0W+vvqd2bBep1HvzjezQ6QSko/6GFTI+WPTlhlk4+Y4Y ldEBCCsnF7AeKTD2gZAdUNQeOITvHZ1HCsj+kzwyIUhSYMt2ri35qfHOexXzUAsjXshU xFc6kfQfSPBktV1r+4fAGS4dne9LPUwpXvkKJ67AwDgU/r7duvM75siDQWc1Ev2k7JEN cA/NrRHVKdzsNijtH01+n7B8XruHctDDhynMzu4+wr55ywJsNsuSayeT6GBhfYnyJbGz MWsw== MIME-Version: 1.0 X-Received: by 10.182.205.200 with SMTP id li8mr12871167obc.18.1442856212290; Mon, 21 Sep 2015 10:23:32 -0700 (PDT) Received: by 10.76.25.66 with HTTP; Mon, 21 Sep 2015 10:23:32 -0700 (PDT) In-Reply-To: References: Date: Mon, 21 Sep 2015 22:53:32 +0530 Message-ID: Subject: Re: Yarn ResourceManager web UI does not show job From: Varun Saxena To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11c2c562fefe200520452336 --001a11c2c562fefe200520452336 Content-Type: text/plain; charset=UTF-8 No, you cant show them in RM UI then. However if you can start another daemon, you can consider using YARN Application History/Timeline Server or MR Job History Server(only for MR jobs) to see information about completed jobs. You can look up Hadoop documentation to learn more about them and how to configure them. Just to clarify though, the apps themselves are not lost, as in, the output is not lost. Its just the information about them which is no longer present on RM restart. Regards, Varun Saxena. On Mon, Sep 21, 2015 at 10:31 PM, Boyu Zhang wrote: > Thanks for the answer Varun. > > It is the case that yarn.resourcemanager.recovery.enabled is set to be > false. Is there a way to show the jobs that are submitted before the > restart? We don't want to lose that data. > > Thanks, > Boyu > > > On Mon, Sep 21, 2015 at 12:53 PM, Varun Saxena > wrote: > >> Hi Boyu, >> >> RM stores apps in state store if recovery is enabled. Only then they will >> be available on restart. >> Otherwise they are kept in memory and hence lost on restart. >> >> You may not have it enabled. Check config value for below config. By >> default its false. >> yarn.resourcemanager.recovery.enabled >> >> Regards, >> Varun. >> >> On Mon, Sep 21, 2015 at 10:01 PM, Boyu Zhang >> wrote: >> >>> Hello Everyone, >>> >>> I have a strange error regarding the ResourceManager web UI ( >>> http://xx.xx:8088). >>> >>> Someone before me set up the hadoop + yarn cluster using Pivotal HD, it >>> was running fine. Then today, the resource manager and node manager >>> disappeared, the logs did not record this. I restarted them, they are up >>> and running, but the resource manger web UI does not show any jobs. We have >>> 700+ jobs in the past, and they were showing before. >>> >>> If I submit MapReduce jobs, the new submitted ones show up. But the >>> disappear again after restart the resource manger and node manager. >>> >>> Can anyone give any hint on where to look? >>> >>> Thanks in advance, >>> Boyu >>> >>> >> > --001a11c2c562fefe200520452336 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
No, you cant show them in RM UI then.

H= owever if you can start another daemon, you can consider using YARN Applica= tion History/Timeline Server or=C2=A0MR Job History Server(only for MR jobs= ) =C2=A0to see information about completed jobs.
You can look up = Hadoop documentation to learn more about them and how to configure them.

Just to clarify though, the apps themselves are not = lost, as in, the output is not lost. Its just the information about them wh= ich is no longer present on RM restart.

Regards,
Varun Saxena.

On Mon, Sep 21, 2015 at 10:31 PM, Boyu Zhang <boyuzha= ng35@gmail.com> wrote:
Thanks for the answer Varun.

It is the case= that=C2=A0yarn.resourcemanager.recovery.enabled is set to be false. Is the= re a way to show the jobs that are submitted before the restart? We don'= ;t want to lose that data.

Thanks,
Boyu<= /div>


On Mon, Sep 21, 2015 at 12= :53 PM, Varun Saxena <vsaxena.varun@gmail.com> wrote:<= br>
Hi Boyu,

<= div>RM stores apps in state store if recovery is enabled. Only then they wi= ll be available on restart.
Otherwise they are kept in memory and= hence lost on restart.

You may not have it enable= d. Check config value for below config. By default its false.


On Mon, Sep 21, 2015 at 10:01 PM, Boyu Zhang <= boyuzhang35@gmail.com> wrote:
Hello Everyone,

I have a strange err= or regarding the ResourceManager web UI (http://xx.xx:8088).

Someone befo= re me set up the hadoop + yarn cluster using Pivotal HD, it was running fin= e. Then today, the resource manager and node manager disappeared, the logs = did not record this. I restarted them, they are up and running, but the res= ource manger web UI does not show any jobs. We have 700+ jobs in the past, = and they were showing before.

If I submit MapReduc= e jobs, the new submitted ones show up. But the disappear again after resta= rt the resource manger and node manager.

Can anyon= e give any hint on where to look?

Thanks in advanc= e,
Boyu




--001a11c2c562fefe200520452336--