Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D245310E17 for ; Mon, 12 Aug 2013 01:28:50 +0000 (UTC) Received: (qmail 71645 invoked by uid 500); 12 Aug 2013 01:28:45 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 71532 invoked by uid 500); 12 Aug 2013 01:28:45 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 71525 invoked by uid 99); 12 Aug 2013 01:28:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Aug 2013 01:28:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com designates 209.85.217.178 as permitted sender) Received: from [209.85.217.178] (HELO mail-lb0-f178.google.com) (209.85.217.178) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Aug 2013 01:28:40 +0000 Received: by mail-lb0-f178.google.com with SMTP id z5so4368191lbh.37 for ; Sun, 11 Aug 2013 18:28:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=T7IpLTZ8JfLXkVJ6jifZfa+UGBtFFJ5WsUhQTP081JI=; b=CFpK+onHPJIC1QY9wxGwc7CB1V6qo2glEM9kbR4Dr6XXZPDs7wQawyAoxZI2dsEiN+ NLpe78m9NEtsShy3xhKysHmP3Q1ctl0/whgIZfZW+KDAEhPm+dlM7In1yd2yGHP9OBzB CHlrEpzth+1y6fYvKrbg+1ro+8NsreZVYPLBbmlB7BcpMXn3WF3XuqO5CcRnEd9I80rX 0jsvdfWOkc3fQivTzr055N3g8JWJhzVULFhiMETIh2r7kAxnP0WhIXDN2Rv/DtvYzERc pXxVw+dGCloZtBiEPbrgfBgqAl03QzWO7dSELg+w6LiFBiuc11YqHa+G0B4GqyHLAQV1 QH0Q== MIME-Version: 1.0 X-Received: by 10.112.55.173 with SMTP id t13mr8768562lbp.39.1376270898652; Sun, 11 Aug 2013 18:28:18 -0700 (PDT) Received: by 10.112.129.40 with HTTP; Sun, 11 Aug 2013 18:28:18 -0700 (PDT) In-Reply-To: References: Date: Sun, 11 Aug 2013 18:28:18 -0700 Message-ID: Subject: Re: DefaultResourceCalculator class not found, ResourceManager fails to start. From: Ted Yu To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a1133e330076dda04e3b60a0b X-Virus-Checked: Checked by ClamAV on apache.org --001a1133e330076dda04e3b60a0b Content-Type: text/plain; charset=ISO-8859-1 Can you check the config entry for yarn.scheduler.capacity.resource-calculator ? It should point to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator bq. I was able to fix all issues What other issues came up ? Thanks On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah wrote: > Hi again > > From a little investigation I have performed I have observed the > following. I assume the module responsible for this class is > hadoop-yarn-common. > > During RM init it crashes since it is looking for a class > DefaultResourceCalculator in > org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator, > while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but > under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus > the RM crashes. Anybody knows how I can fix this? I would very grateful for > any help. > > regards > tmp > > > 2013/8/11 Rob Blah > >> Hi >> >> I have a strange problem, regarding missing class, the >> DefaultResourceCalculator. I have a single node sandbox cluster working in >> a pseudo-distributed mode. The cluster was working fine yesterday, however >> today it stopped working. I was able to fix all issues except the following >> problem in ResourceManager: >> 2013-08-11 12:12:42,425 FATAL >> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error >> starting ResourceManager >> java.lang.RuntimeException: java.lang.RuntimeException: >> java.lang.ClassNotFoundException: Class >> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator >> not found >> at >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753) >> at >> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333) >> at >> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258) >> at >> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241) >> at >> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) >> at >> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826) >> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: >> Class >> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator >> not found >> at >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721) >> at >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745) >> ... 5 more >> Caused by: java.lang.ClassNotFoundException: Class >> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator >> not found >> at >> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1625) >> at >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1719) >> ... 6 more >> 2013-08-11 12:12:42,426 INFO >> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: >> SHUTDOWN_MSG: >> >> I build YARN from sources, daily updated to the newest revision in trunk. >> The class DefaultResourceCalculator exists and is present in YARN's >> sources. I am using (currently) trunk revision 1512895. I build YARN >> project with the following command: >> mvn clean package -Pdist -Dtar -DskipTests >> I create tar ball with the use of script provided in the sources: >> dist-tar-stitching.sh (hadoop-dist/target) >> >> regards >> tmp >> > > --001a1133e330076dda04e3b60a0b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Can you check the config entry for=A0yarn.scheduler.capacity.resource-calcu= lator ?
It should point to=A0org.apache.hadoop.yarn.util.resource.Defau= ltResourceCalculator

bq.=A0I was able to fix all i= ssues

What other issues came up ?

Th= anks

On Sun, Aug 11, 2013 at 2:07 PM, Rob= Blah <tmp5330@gmail.com> wrote:
Hi again
From a little investigation I have performed I have observed the follo= wing. I assume the module responsible for this class is hadoop-yarn-common.=

During RM init it crashes since it is looking for a class Default= ResourceCalculator in org.apache.hadoop.yarn.server.resourcemanager.resourc= e.DefaultResourceCalculator, while the class is present in hadoop-yarn-comm= on-3.0.0-SNAPSHOT.jar but under org.apache.hadoop.yarn.util.resource.Defaul= tResourceCalculator. Thus the RM crashes. Anybody knows how I can fix this?= I would very grateful for any help.

regards
tmp


=
2013/8/11 Rob Blah <tmp5330@gmail.com>
Hi
<= br>
I have a strange problem, regarding missing class, the DefaultReso= urceCalculator. I have a single node sandbox cluster working in a pseudo-di= stributed mode. The cluster was working fine yesterday, however today it st= opped working. I was able to fix all issues except the following problem in= ResourceManager:
2013-08-11 12:12:42,425 FATAL org.apache.hadoop.yarn.server.resourcemanager= .ResourceManager: Error starting ResourceManager
java.lang.RuntimeExcept= ion: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class or= g.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalcula= tor not found
=A0=A0=A0 at org.apache.hadoop.conf.Configuration.getClass(Configuration.ja= va:1753)
=A0=A0=A0 at org.apache.hadoop.yarn.server.resourcemanager.sche= duler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(Capacit= ySchedulerConfiguration.java:333)
=A0=A0=A0 at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capaci= ty.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
=A0=A0=A0 = at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceIni= t(ResourceManager.java:241)
=A0=A0=A0 at org.apache.hadoop.service.AbstractService.init(AbstractService= .java:163)
=A0=A0=A0 at org.apache.hadoop.yarn.server.resourcemanager.Re= sourceManager.main(ResourceManager.java:826)
Caused by: java.lang.Runtim= eException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.= server.resourcemanager.resource.DefaultResourceCalculator not found
=A0=A0=A0 at org.apache.hadoop.conf.Configuration.getClass(Configuration.ja= va:1721)
=A0=A0=A0 at org.apache.hadoop.conf.Configuration.getClass(Conf= iguration.java:1745)
=A0=A0=A0 ... 5 more
Caused by: java.lang.ClassN= otFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resou= rce.DefaultResourceCalculator not found
=A0=A0=A0 at org.apache.hadoop.conf.Configuration.getClassByName(Configurat= ion.java:1625)
=A0=A0=A0 at org.apache.hadoop.conf.Configuration.getClas= s(Configuration.java:1719)
=A0=A0=A0 ... 6 more
2013-08-11 12:12:42,4= 26 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUT= DOWN_MSG:

I build YARN from sources, daily updated to the newest revision i= n trunk. The class DefaultResourceCalculator exists and is present in YARN&= #39;s sources. I am using (currently) trunk revision 1512895. I build YARN = project with the following command:
mvn clean package -Pdist -Dtar -DskipTests
I create tar ball with = the use of script provided in the sources:
dist-tar-stitching.sh (hadoop= -dist/target)

regards
tmp


--001a1133e330076dda04e3b60a0b--