Return-Path: X-Original-To: apmail-hadoop-yarn-dev-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 655A510525 for ; Mon, 31 Mar 2014 23:21:06 +0000 (UTC) Received: (qmail 77342 invoked by uid 500); 31 Mar 2014 23:21:05 -0000 Delivered-To: apmail-hadoop-yarn-dev-archive@hadoop.apache.org Received: (qmail 77289 invoked by uid 500); 31 Mar 2014 23:21:05 -0000 Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-dev@hadoop.apache.org Delivered-To: mailing list yarn-dev@hadoop.apache.org Received: (qmail 77281 invoked by uid 99); 31 Mar 2014 23:21:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Mar 2014 23:21:04 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of azuryyyu@gmail.com designates 209.85.160.44 as permitted sender) Received: from [209.85.160.44] (HELO mail-pb0-f44.google.com) (209.85.160.44) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Mar 2014 23:21:00 +0000 Received: by mail-pb0-f44.google.com with SMTP id rp16so8903767pbb.31 for ; Mon, 31 Mar 2014 16:20:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:mime-version:subject :message-id:date:references:in-reply-to:to; bh=5ri5b0bBQBO6/LtV+a5pSwzyOZTx72ZxKMfL+a6FBPc=; b=CMWfTmOcjnesxasvYRbQ6MBehJVG8qI0Xxptt4vdVAUMZ2pVm6XPFkZQG16+vtIi17 e67lW1qB6vs8jI0ze4jD/XXUimImAQZx7HOsMgqsrLnLjCnUY34Yn67fsVeX1NhYud83 6g3w9I6JrQnhH0M54TYxgPfZiMfWKbtAA4NCkUrQqdI8fwknHm7/RHJ8MQePtOfzVA1d BIHH9BLCqxOfM8iDr6k1W8SbOzp5QHCsWJ14o0YQYwKRX6hTyMre3K4GG3K6HfG68FVM QQI69yK8TnyJ05cdPW23GrAGY3b0DFmnroVptXgSAvjhynjhfZnxIWGFeWlOct0E2mX2 DPTw== X-Received: by 10.68.244.229 with SMTP id xj5mr28076927pbc.108.1396308039658; Mon, 31 Mar 2014 16:20:39 -0700 (PDT) Received: from [192.168.1.100] ([106.120.53.207]) by mx.google.com with ESMTPSA id te2sm46683051pac.25.2014.03.31.16.20.37 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 31 Mar 2014 16:20:38 -0700 (PDT) From: Azuryy Content-Type: text/plain; charset=gb2312 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Subject: Re: RM ha issuses Message-Id: Date: Tue, 1 Apr 2014 07:20:32 +0800 References: In-Reply-To: To: "yarn-dev@hadoop.apache.org" X-Mailer: iPhone Mail (11D167) X-Virus-Checked: Checked by ClamAV on apache.org I will run a MR job to verify it. Stop RM means yarn-daemon.sh stop resourcemanager=20 Thanks Sent from my iPhone5s > On 2014=C4=EA4=D4=C21=C8=D5, at 0:38, Karthik Kambatla wrote: >=20 > Thanks for reporting this, Azuryy. Indeed, this is surprising. >=20 > I don't quite understand how Hive works; do you mind running a vanilla MR > job and verifying if this is indeed the case. Also, when you say you > stopped the Active RM, you mean only the RM process - correct? >=20 >=20 >> On Mon, Mar 31, 2014 at 3:46 AM, Azuryy Yu wrote: >>=20 >> Hi, >>=20 >> I built from trunk, and configured RM Ha, then I submitted a hive job. >> total 11 maps, then I stopped active RM when 6 maps finished. >>=20 >> but Hive shows me all map tasks restat again. This is conflict with the >> design description. >>=20 >> job progress: >> 2014-03-31 18:44:14,088 Stage-1 map =3D 68%, reduce =3D 0%, Cumulative C= PU >> 713.84 sec >> 2014-03-31 18:44:15,128 Stage-1 map =3D 68%, reduce =3D 0%, Cumulative C= PU >> 722.83 sec >> 2014-03-31 18:44:16,160 Stage-1 map =3D 68%, reduce =3D 0%, Cumulative C= PU >> 731.95 sec >> 2014-03-31 18:44:17,191 Stage-1 map =3D 68%, reduce =3D 0%, Cumulative C= PU >> 744.17 sec >> 2014-03-31 18:44:18,220 Stage-1 map =3D 68%, reduce =3D 0%, Cumulative C= PU >> 756.22 sec >> 2014-03-31 18:44:19,250 Stage-1 map =3D 68%, reduce =3D 0%, Cumulative C= PU >> 762.4 sec >> 2014-03-31 18:44:20,281 Stage-1 map =3D 68%, reduce =3D 0%, Cumulative C= PU >> 774.64 sec >> 2014-03-31 18:44:21,306 Stage-1 map =3D 70%, reduce =3D 0%, Cumulative C= PU >> 786.49 sec >> 2014-03-31 18:44:22,334 Stage-1 map =3D 70%, reduce =3D 0%, Cumulative C= PU >> 792.59 sec >> 2014-03-31 18:44:23,363 Stage-1 map =3D 73%, reduce =3D 0%, Cumulative C= PU >> 807.58 sec >> 2014-03-31 18:44:24,392 Stage-1 map =3D 77%, reduce =3D 0%, Cumulative C= PU >> 815.96 sec >> 2014-03-31 18:44:25,416 Stage-1 map =3D 80%, reduce =3D 0%, Cumulative C= PU >> 823.83 sec >> 2014-03-31 18:44:26,443 Stage-1 map =3D 80%, reduce =3D 0%, Cumulative C= PU >> 826.84 sec >> 2014-03-31 18:44:27,472 Stage-1 map =3D 82%, reduce =3D 0%, Cumulative C= PU >> 832.16 sec >> 2014-03-31 18:44:28,501 Stage-1 map =3D 84%, reduce =3D 0%, Cumulative C= PU >> 839.73 sec >> 2014-03-31 18:44:29,531 Stage-1 map =3D 86%, reduce =3D 0%, Cumulative C= PU >> 844.45 sec >> 2014-03-31 18:44:30,564 Stage-1 map =3D 82%, reduce =3D 0%, Cumulative C= PU >> 760.34 sec >> 2014-03-31 18:44:31,728 Stage-1 map =3D 0%, reduce =3D 0% >> 2014-03-31 18:45:06,918 Stage-1 map =3D 2%, reduce =3D 0%, Cumulative CP= U >> 213.81 sec >> 2014-03-31 18:45:07,952 Stage-1 map =3D 2%, reduce =3D 0%, Cumulative CP= U >> 216.83 sec >> 2014-03-31 18:45:08,979 Stage-1 map =3D 7%, reduce =3D 0%, Cumulative CP= U >> 229.15 sec >> 2014-03-31 18:45:10,007 Stage-1 map =3D 11%, reduce =3D 0%, Cumulative C= PU >> 244.42 sec >> 2014-03-31 18:45:11,040 Stage-1 map =3D 14%, reduce =3D 0%, Cumulative C= PU >> 247.31 sec >> 2014-03-31 18:45:12,072 Stage-1 map =3D 18%, reduce =3D 0%, Cumulative C= PU >> 259.5 sec >> 2014-03-31 18:45:13,105 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 274.72 sec >> 2014-03-31 18:45:14,135 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 280.76 sec >> 2014-03-31 18:45:15,170 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 292.9 sec >> 2014-03-31 18:45:16,202 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 305.16 sec >> 2014-03-31 18:45:17,233 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 314.21 sec >> 2014-03-31 18:45:18,264 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 323.34 sec >> 2014-03-31 18:45:19,294 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 335.6 sec >> 2014-03-31 18:45:20,325 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 344.71 sec >> 2014-03-31 18:45:21,355 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 353.8 sec >> 2014-03-31 18:45:22,385 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 366.06 sec >> 2014-03-31 18:45:23,415 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 375.2 sec >> 2014-03-31 18:45:24,449 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 384.28 sec >> 2014-03-31 18:45:25,481 Stage-1 map =3D 23%, reduce =3D 0%, Cumulative C= PU >> 396.54 sec >> 2014-03-31 18:45:26,512 Stage-1 map =3D 25%, reduce =3D 0%, Cumulative C= PU >> 408.72 sec >> 2014-03-31 18:45:27,549 Stage-1 map =3D 25%, reduce =3D 0%, Cumulative C= PU >> 414.69 sec >> 2014-03-31 18:45:28,582 Stage-1 map =3D 30%, reduce =3D 0%, Cumulative C= PU >> 426.99 sec >> 2014-03-31 18:45:29,614 Stage-1 map =3D 32%, reduce =3D 0%, Cumulative C= PU >> 439.25 sec >> 2014-03-31 18:45:30,653 Stage-1 map =3D 34%, reduce =3D 0%, Cumulative C= PU >> 448.25 sec >> 2014-03-31 18:45:31,683 Stage-1 map =3D 39%, reduce =3D 0%, Cumulative C= PU >> 460.5 sec >> 2014-03-31 18:45:32,723 Stage-1 map =3D 41%, reduce =3D 0%, Cumulative C= PU >> 469.63 sec >> 2014-03-31 18:45:33,754 Stage-1 map =3D 43%, reduce =3D 0%, Cumulative C= PU >> 478.67 sec >>=20